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ABSTRACT 


People  gain  varying  degrees  of  expertise  in  their  field  of  work. 
Criteria  within  identified  domains  must  be  clearly  defined  if  we  are 
to  assess  degree  of  expertise  and  identify  differences  between  the 
skills  of  novices  and  experts. 

This  research  included  the  identification  of  skills  required  to 
interpret  from  English  to  American  Sign  Language  (ASL)  through 
reviewing  the  available  literature  and  conducting  task  and  error 
analyses  of  actual  interpretation  samples.  Eight  Major  Features  of 
interpretation  were  identified.  Each  Major  Feature  contained  lists 
of  related  skills  with  each  skill  followed  by  lists  of  relevant  and 
representative  errors.  In  addition,  definitions  were  added  to  further 
describe  each  skill. 

Following  the  initial  identification  of  skills,  a  criterion- 
referenced  diagnostic  instrument  was  developed  to  assess  English- 
to-ASL  interpretations.  The  instrument  was  validated  using  24  Deaf 
and  hearing  experts  from  across  North  America  to  conduct  a  panel 
review  and  two  field  tests.  During  the  field  tests,  the  experts  used 
the  instrument  to  assess  two  video  taped  interpretation  samples. 

The  instrument  was  revised  after  each  validation  stage. 

The  results  of  the  research  are  presented  as  a  draft  diagnostic 
assessment  instrument  and  a  rater's  instruction  manual  for 
assessing  English  to  ASL  interpretations.  These  documents  can  be 
used  to  conduct  ongoing  research  related  to  assessing  interpreting 
skills.  Further  development  of  the  instrument  must  continue. 

The  findings  showed  that  there  was  slightly  less  than  100% 
consistency  of  agreement  within  each  rater.  The  overall  consistency 
of  agreement  among  raters  on  two  interpretation  samples  was 
higher  at  the  error  level,  with  77%  and  95%,  than  at  the  skill  level, 
with  64%  and  54%,  suggesting  that  rating  the  skills  without  rating 
the  errors  is  less  consistent  than  rating  at  the  error  level. 

Implications  for  practice  as  well  as  suggestions  for  future 
research  are  discussed. 
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CHAPTER  1 :  INTRODUCTION  AND  BACKGROUND  OF  THE  PROBLEM 

Communication 

Communication  is  essential  to  human  existence.  In  its 
broadest  and  most  common  use  it  can  be  verbal  or  nonverbal.  Verbal 
communication  is  characterized  by  the  expression  of  thoughts 
through  words  in  an  oral-aural  modality  (Hayakawa,  1978).  These 
verbal  communications  are  conveyed  using  a  variety  of  techniques 
such  as  manipulation  of  intonation,  enunciation,  pronunciation,  and 
voice  quality  in  speech.  In  contrast,  characteristics  and  behaviors 
exhibited  through  posture,  eye  contact,  facial  expression,  and 
nuances  of  the  body  and  face  constitute  nonverbal  communication. 

Communication  is  needed  in  order  to  relay  information  from 
one  person  to  another  person  or  from  one  group  of  people  to  another 
group  of  people.  Communication  is  a  strategy  by  which  people 
exchange  ideas  and  opinions.  It  is  a  method  through  which  one  can 
express  feelings  and  thoughts  both  to  oneself  and  to  others. 

Humans  need  to  communicate  in  order  to  function  in  the 
society  in  which  they  live.  In  many  countries  around  the  world, 
communication  skills  are  necessary  to  be  effective  employees  and 
effective  employers.  Often  job  announcements  specify  that 
effective  communication  skills  are  essential  for  certain  positions. 

In  education  a  person  must  communicate  ideas  via  spoken  and 
written  forms  in  order  to  advance  successfully  from  one  grade  or 
academic  level  to  the  next. 

Without  communication  people  are  isolated  and  not  able  to 
understand  what's  happening  around  them.  Consequently,  people 
lacking  communication  skills  may  be  behind  their  peers  in  social 
interaction  and  educational  achievements.  In  many  societies,  a 
person  with  poor  communication  skills  may  be  disempowered; 
empowerment  will  belong  with  those  who  communicate  well. 

Communication  can  take  place  using  other  methods  or  systems, 
for  example,  Morse  Code  and  the  use  of  flags.  However,  language  is 
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one  of  the  most  common  forms  of  communication.  The  commonly 
held  definition  of  a  language  is  that  it  is  aural-oral,  heard  and 
spoken,  and  is  represented  by  a  symbol  system  of  sounds  used  to 
symbolize  meaning  in  the  world  and  meaning  to  each  other 
(Hayakawa,  1978). 

According  to  Stokoe  (1973),  a  linguist,  this  definition  fails  to 
include  visual-manual  languages,  that  is,  sign  languages.  It  has  been 
well  documented  over  the  past  15  years  that  sign  languages  are 
indeed  bona  fide  languages  (e.g.,  Baker  &  Cokely,  1980;  Klima  & 
Bellugi,  1 979;  Woodward,  1978).  Like  any  language,  sign  language 
has  its  own  lexicon,  syntactic  structure,  and  grammar  to  convey 
meaning.  It  is  based  on  a  visual-manual  mode  of  communication  and 
is  used  primarily  by  culturally  Deaf  individuals,  that  is,  those  who 
are  raised  in  an  environment  exposed  to  sign  language  through  Deaf 
adults  who  pass  on  their  language  and  their  culture  from  generation 
to  generation  (Padden,  1980). 

There  are  many  different  sign  languages  (Frishberg,  1990),  for 
example,  American  Sign  Language,  British  Sign  Language,  Japanese 
Sign  Language,  and  German  Sign  Language.  The  notion  that  sign 
language  is  a  universal  language  is  a  myth.  People  who  use  sign 
language  to  communicate  use  different  languages.  Usually  these 
people  are  either  Deaf  or  friends  or  relatives  of  Deaf  persons  who 
through  necessity  and/or  opportunity  have  acquired  the  sign 
language  used  in  their  Deaf  community. 

Before  discussing  communication  between  hearing  and  Deaf 
individuals,  the  issue  of  bridging  the  gap  between  people  who  do  not 
share  the  same  language  will  be  briefly  examined,  that  is, 
interpretation. 


Interpretation 

The  term  interpretation  has  several  different  meanings. 
Interpretation  can  mean  interpreting  music,  a  theatrical 
performance,  someone's  nonverbal  communication,  and  even 
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environmental  interpreting  in  a  park.  Such  interpretation  is  from  an 
affective,  abstract,  and  subjective  viewpoint,  and  is  different  from 
interpretation  for  the  purpose  of  communication  among  people. 

Within  the  interpreting  profession,  the  term  interpretation  is 
also  distinct  from  the  term  'translation'.  There  are  many  different 
types  of  translation.  The  most  common  type  refers  to  the  written 
text  of  one  language  being  translated  into  a  written  text  of  another 
language.  The  key  element  is  that  translation  is  written  and  not 
spoken  (Seleskovitch,  1978).  In  contrast,  interpretation  is  spoken. 

It  is  the  facilitation  of  communication  between  two  or  more  people 
who  do  not  share  the  same  language. 

The  Interpreting  Process 

Consecutive  and  simultaneous  interpreting  are  two  commonly 
used  methods  of  interpretation.  Consecutive  interpretation  refers 
to  the  process  of  listening  to  the  speaker  and  waiting  until  a  few 
sentences,  a  paragraph,  or  an  entire  speech  is  rendered.  Then,  either 
by  memory  if  the  passage  was  short,  or  by  using  notes  if  the  passage 
was  long  or  technical,  the  interpreter  relays  the  information  into 
the  second  language. 

Simultaneous  interpretation,  the  type  of  interpreting  most 
often  used  in  the  United  Nations  (Frishberg,  1990)  and  other 
meetings  of  this  caliber,  is  the  process  by  which  interpreters  are 
only  seconds  behind  the  original  speaker  in  their  interpretation  of 
the  message  into  a  second  language. 

The  process  of  interpretation  is  complex.  It  entails  the  ability 
to  move  from  one  language  to  another  language  with  ease,  comfort, 
and  accuracy.  Interpreters  must  listen  to  the  incoming  message  in 
the  source  language,  process  this  message,  and  then  articulate  it 
into  an  equivalent  message  in  the  target  language  while  often 
simultaneously  listening  to  further  incoming  messages. 

Interpreters  must  have  the  ability  to  successfully  accomplish 
multi-task  mental  processing. 
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In  addition  to  the  mental  processing  of  one  language  into 
another,  the  interpreter  must  be  able  to  quickly  and  smoothly 
interpret  from  the  source  language  into  the  target  language  and  back 
again  when  the  conversation  is  not  one  way  in  direction  but  back  and 
forth.  When  transmitting  the  information  from  one  party  to  another, 
interpreters  are  also  required  to  do  it  as  faithfully  as  possible  and 
in  a  manner  which  reflects  the  mood,  affect,  and  nuances  of  all 
parties  involved. 

When  one  discusses  interpretation  of  language  for  purposes  of 
communication,  it  is  not  simply  the  transmission  of  vocabulary  or 
sentences  or  even  concepts;  it  also  includes  conveying  the  cultural 
nuances  associated  with  both  languages.  Language  actually  shapes 
people's  perception  and  thinking,  thus  affecting  their  actions 
(Condon  &  Yousef,  1984).  People  belonging  to  a  particular  culture 
use  their  own  language  when  talking  to  people  from  their  cultural 
community.  “A  culture  is  a  set  of  learned  behaviors  of  a  group  of 
people  who  have  their  own  language,  values,  rules  for  behavior,  and 
traditions"  (Padden,  1980,  p.  92).  Cultural  information  must  be 
conveyed  and  cultural  adjustments  appropriately  used  in  the 
interpretation  process.  An  interpreter  in  effect  is  a  mediator  of 
both  language  and  culture.  Interpreters  are  not  only  bilingual  but 
also  bicultural.  Because  language  and  culture  are  so  closely  tied  (in 
fact  at  times  impossible  to  separate),  often  an  interpreter  must 
include  both  in  the  interpretation  process.  The  interpreter  who  is 
enculturated  in  both  language  communities  can  make  these 
adjustments  and  focus  on  the  mental  processes  necessary  to  render 
a  competent  interpretation. 

An  interpretation,  like  a  translation,  is  not  a  word  for  word,  or 
a  concept  for  concept  articulation  of  a  message.  It  is  an  entire 
process  with  the  goal  of  equating,  as  much  as  is  possible,  a  message 
from  one  person's  language  into  the  language  of  another  person,  so 
that  the  person  receiving  the  interpretation  is  affected  as  if  the 
message  was  heard  in  its  original  language. 
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History  of  Spoken  Language  Interpreting 

Whenever  people  from  two  different  language  groups  come 
together  in  order  to  communicate,  they  must  negotiate  using  either  a 
common  language  or  a  common  system  by  which  they  can  make 
themselves  understood.  When  people  do  not  share  the  same  language, 
one  way  of  negotiating  this  communication  barrier  is  to  employ  the 
services  of  an  interpreter  who  is  fluent  in  both  languages. 

After  World  War  I,  representatives  of  nations  had  a  dire  need 
to  communicate  with  each  other  on  an  international  and  political 
level.  There  was  a  need  for  qualified  interpreters  who  could 
interpret  at  a  sophisticated  level  and  handle  the  nuances  that 
prevail  in  politics.  As  well,  interpreters  needed  to  be  adept  at  more 
than  one  language  in  order  to  carry  on  international  business. 
Professional  spoken  language  interpreters  emerged  as  a  result  of 
international  meetings  and  the  need  for  shared  communication 
among  representatives  from  different  nations  (Frishberg,  1990). 

Prior  to  this  time  there  were  always  people  who  acted  in  an 
informal  capacity  conveying  messages  between  parties  who  wished 
to  express  themselves  but  lacked  the  ability  to  use  the  other's 
language  (Seleskovitch,  1978).  For  example,  minority  language 
groups  within  a  dominant  culture  who  wanted  to  speak  with  people 
of  the  majority  language  group  (or  vice  versa)  needed  to  have  an 
'interpreter'  to  convey  messages.  These  situations  encompassed 
everything  from  banking  needs  and  filling  out  employment 
application  forms  to  making  funeral  arrangements  for  deceased 
family  members.  Often  the  individuals  who  performed  this 
'interpreting'  task  were  children  or  other  family  members  who  spoke 
the  language  of  the  minority  language  group  and  who  had  an 
opportunity  to  acquire  the  language  of  the  majority  language  group. 
They  acted  as  the  'go-between'  or  the  'interpreter'  in  situations 
where  it  was  needed. 
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Deafness 

Many  hard  of  hearing  people,  having  only  a  mild  to  moderate 
hearing  loss,  function  within  their  community  using  the  sense  of 
hearing.  These  individuals  include  those  who  were  born  with  a 
hearing  loss  and  those  who  lost  some  of  their  hearing  later  in  life, 
after  acquiring  speech.  On  the  other  hand,  there  are  those 
individuals  who  were  either  born  deaf  or  were  deafened  at  a  very 
young  age  prior  to  learning  a  spoken  language.  Many  of  the  latter  are 
educated  in  residential  schools  and  socialize  within  the  Deaf 
community,  thereby  learning  sign  language  as  their  primary  mode  of 
communication,  instead  of  a  spoken  language  which  depends  on  the 
sense  of  hearing  (Padden  &  Humphries,  1988). 

Sign  Languages 

In  Canada  and  the  United  States  at  least  two  sign  languages 
are  prevalently  used:  la  Langue  des  Signes  Quebecoise  (LSQ)  and 
American  Sign  Language  (ASL).  In  the  province  of  Quebec  and  other 
nearby  French  speaking  communities,  Deaf  people  use  LSQ,  which  is 
distinct  from  ASL,  just  as  spoken  French  is  distinct  from  spoken 
English. 

In  North  America,  ASL  is  used  with  only  slight  variations 
among  different  regions.  Regional  differences  exist  in  lexicon, 
speed,  and  signing  style  much  like  that  of  spoken  English  (Sternberg, 
1981).  For  example,  there  are  a  number  of  ASL  regional  lexical 
differences  to  express  the  meanings  for  'birthday'  and  'early'.  Also, 
signing  speed  can  vary  regionally  much  like  the  speed  with  which 
people  speaking  English  vary  from  the  east  to  the  west  coast  of  the 
United  States. 


7 


Deaf  Culture  Interacting  with  Hearing  Culture 

Deaf  people  have  their  own  distinct  culture  with  their  own 
sign  languages,  values,  social  norms,  rules  for  behavior,  and 
traditions.  This  culture  is  different  from  the  mainstream  culture  in 
which  they  live.  Thus,  the  word  'Deaf  is  capitalized  to  indicate  a 
cultural  group  of  people.  In  contrast  'deaf'  is  used  when  referring  to 
people  who  have  a  loss  of  hearing  but  are  not  a  part  of  a  Deaf 
cultural  group. 

For  the  Deaf  community,  this  culture  is  learned  by  the  minority 
living  within  the  majority  hearing  group.  Many  Deaf  people  are 
bicultu ral ,  meaning  they  have  acquired  the  knowledge  and  skills  to 
move  with  differing  degrees  of  ease  between  and  among  hearing  and 
Deaf  cultures.  Deaf  people  may  attend  hearing  functions.  They  may 
attend  colleges  and  universities  with  hearing  people.  Their 
colleagues  at  work  may  be  hearing.  The  majority  culture  in  which 
they  live  is  hearing. 

One  major  difference  exists  between  Deaf  people  as  a  minority 
group  and  other  minority  groups.  Deaf  people  do  not  have  a  sense  of 
hearing.  Therefore,  they  can  not  learn  naturally  the  spoken  language 
used  in  their  society.  There  are  rare  instances  where  a  Deaf  person 
has  an  exceptional  talent  to  speechread  and/or  speak,  but  even  this 
is  learned  artificially  and  not  naturally  as  a  spoken  language  is 
learned. 


Sign  Language  Interpreters 
History 

Sign  Language  interpretation  began  when  hearing  people 
started  working  with  Deaf  individuals  within  their  own  family  unit 
(Pimentel,  1979).  Initially,  friends  and  family  members  tended  to 
act  as  'interpreters',  or  more  accurately  'helpers',  when  the  Deaf 
person  needed  to  communicate  with  hearing  people  to  share 
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information.  Often  hearing  children  were  used  as  'helpers'  to  bridge 
the  gap  between  their  Deaf  parents  and  the  hearing  world  at  large 
(Frishberg,  1990).  This  important  role  of  'helper'  also  became 
popular  in  the  religious  sector  where  religious  workers  learned  sign 
language  to  educate  and  introduce  Deaf  people  to  religion  (Frishberg, 
1990). 

In  the  United  States,  the  profession  of  ASL/English 
interpreting  was  acknowledged  in  1964  through  a  series  of 
conferences  (Frishberg,  1986).  North  American  national 
associations  of  ASL/English  interpreters  began  in  the  1960's  and 
1970's.  In  the  United  States,  the  Registry  of  Interpreters  for  the 
Deaf  (RID)  was  founded  in  1964  and  incorporated  in  1972.  In  Canada, 
the  Association  of  Visual  Language  Interpreters  of  Canada  (AVLIC) 
was  founded  in  1971  and  incorporated  in  1981. 

In  the  United  States,  as  a  result  of  the  Rehabilitation  Acts  of 
1965  and  1973,  which  required  funds  to  be  used  to  provide 
interpreting  services  for  vocational  rehabilitation  clients  and 
mandated  equal  access  for  disabled  people  to  federally  sponsored 
programs  such  as  employment,  education  and  health,  there  was  an 
increase  in  the  demand  for  sign  language  interpreters  (Frishberg, 
1990). 

Today,  interpreters  are  needed  for  a  multitude  of  daily 
situations  for  both  Deaf  and  hearing  people,  such  as:  the  Deaf 
lawyer  and  a  hearing  judge;  the  Deaf  printer  and  his  employer;  the 
Deaf  parents  of  an  ill  child  in  a  hospital;  the  Deaf  couple  going 
before  a  rabbi  or  minister  for  the  exchange  of  their  marriage  vows; 
the  mainstreamed  Deaf  student,  hearing  teachers,  and  classmates; 
and  the  Deaf  person  who  wants  to  take  a  CPR  course  offered  at  the 
local  community  center. 


Education 

In  the  early  1960's,  it  was  recognized  that  there  was  a 
shortage  of  competent  interpreters  (Fant,  1990).  It  became 
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imperative,  as  a  result  of  the  Rehabilitation  Acts  of  1965  and  1973, 
that  the  United  States  train  skilled  interpreters  to  facilitate 
communication  effectively  between  Deaf  and  hearing  people  so  as  to 
provide  Deaf  people  access  to  federally  sponsored  programs  as 
mandated  by  federal  law. 

Early  in  the  1970's  formal  education  of  interpreters  began  in 
the  United  States  (Neumann  Solow,  1981),  while  in  Canada,  the  first 
ASL/English  interpreter  training  program  was  established  in  1982 
(Schein,  Carver,  &  Mallory,  1989).  Prior  to  this  time,  there  were 
sign  language  programs  offering  courses  in  sign  language,  which 
then  graduated  students  who  worked  in  the  interpreting  field. 
However,  a  program  containing  extensive  work  in  interpreting  was 
not  achieved  in  Canada  until  1982  (Schein  et  al. ,  1989). 

Certification  and  Assessment 

As  a  result  of  professional  training,  more  interpreters  began 
entering  the  work  force.  Once  they  were  in  the  work  force, 
employers,  both  Deaf  and  hearing,  identified  a  need  for  assessment 
to  determine  the  competency  level  of  interpreters. 

In  1964,  at  the  founding  of  RID,  there  was  agreement  that  an 
organization  was  needed  to  assess  interpreters'  competencies  (Fant, 
1990).  Eight  years  later  RID  began  certifying  interpreters  (Fant, 

1990) .  The  Canadian  national  association  of  interpreters,  AVLIC, 
implemented  its  first  certifying  exam  in  August,  1990  (Evans, 

1991) .  At  the  present  time  both  of  these  certifying  examinations 
are  normative-based  and  do  not  provide  diagnostic  feedback  to  the 
interpreters  taking  the  examinations. 

Per-Lee  (1981)  stated  there  is  an  identified  need  to  develop  a 
profile  of  relevant  characteristics  of  sign  language  interpreters. 

Deaf  people  have  intuitively  recognized  that  interpreters  vary 

greatly  in  their  skills.  There  is  no  scientific  body  of 

knowledge,  however,  defining  the  characteristics  of  those 


factors  that  make  a  good  interpreter.  There  is,  at  this  time,  a 
unique  need  of  8,000  interpreters  and  an  increased  projection 
for  the  next  five  years.  Therefore,  research  is  needed  to 
identify  those  characteristics  which  can  be  transferred  into 
professional  development  programs  and  certification 
standards  for  successful  interpreters,  (p.  14) 

Cokely  (1986)  investigated  different  interpretation  models 
that  primarily  were  concerned  with  the  cognitive  processes  involved 
in  the  interpretation  process.  The  source  language  message  was  a 
key  element  in  the  development  and  analysis  of  the  various  models. 
He  found  in  his  research  that  interpreters  must  understand  the 
message  they  are  interpreting  or  else  they  will  fail  to  interpret  it 
accurately. 

Specifically,  Cokely  conducted  research  on  the  relationship  of 
the  amount  of  lag  time  interpreters  have  between  the  utterance  of 
the  source  language  and  the  rendering  of  the  interpreted  message. 
Interpeters  were  video  taped  while  interpreting  for  a  live  spoken 
English  expository  monologue.  Through  a  sampling  procedure  of  one 
minute  out  of  every  five  minutes,  a  total  of  approximately  8  minutes 
of  video  tape  was  used.  The  English  and  the  ASL  was  transcribed  as 
well  as  the  elapsed  time,  down  to  the  tenth  of  a  second. 

He  was  particularly  interested  in  the  interpreters'  processing 
time  and  the  relationship  between  the  length  of  the  mental 
processing  time  used  and  the  number  of  errors  in  the  interpreted 
message.  He  analyzed  the  errors  and  miscues  interpreters  made. 
Deviations  from  the  original  message  were  considered  miscues.  In 
all,  five  major  categories  of  miscues  were  identified:  omissions, 
additions,  substitutions,  intrusions,  and  anomalies.  His  research 
indicated  that  the  less  the  lag  the  higher  the  occurrence  of  miscues. 


Purpose  of  the  Research 


Deaf  people  have  the  right  to  access  the  hearing  community  in 
which  they  live.  One  essential  way  of  providing  this  access  is 
through  the  provision  of  competent  Sign  Language  interpreters.  As 
Per-Lee  (1981)  stated,  there  is  a  need  to  research  the 
characteristics  of  competent  interpreters.  Therefore,  the  purpose 
of  this  research  was  first  to  identify  the  skills  demonstrated  in 
interpretations  from  the  source  language  of  spoken  English  to  the 
target  language  of  American  Sign  Language  (ASL).  Second,  the 
purpose  was  to  develop  a  valid  and  reliable  criterion-referenced 
instrument  for  assessing  interpreters  with  different  levels  of  skills 
when  interpreting  from  English  to  ASL.  The  instrument  could  then  be 
used  for  diagnostic  purposes  by  providing  interpreters  with  an 
assessment  of  their  strengths  and  weaknesses. 

Justification 

Legislation 

Only  by  providing  Deaf  people  with  access  to  the  hearing 
community  will  they  have  the  opportunity  to  achieve  success  equal 
to  that  of  the  hearing  community. 

As  discussed  in  the  previous  sections,  law  requires  that  Deaf 
people  have  access  to  the  services  within  the  hearing  community. 
Federal  legislation  in  the  United  States  and  Canada  concerning 
people  with  disabilities  is  prevalent.  In  addition  to  the 
Rehabilitation  Acts  of  1965  and  1973,  several  other  laws  and 
activities  have  been  mandated.  For  example,  the  United  States 
passed  the  Americans  With  Disabilities  Act  ("Senate  passed  bill. M 
1989).  The  Canadian  federal  government  proclaimed  1983-1992  the 
Decade  of  Disabled  Persons  and  established  the  National  Strategy  for 
the  Integration  of  Persons  with  Disabilities  as  recently  as  1991. 


Accessibility  for  disabled  people  is  an  important  current  issue  in 
North  America. 


The  Interpreting  Profession 

Professional  interpreting  organizations  and  interpreting 
programs  have  an  obligation  to  ensure  that  their  members  and  their 
graduates  are  competent.  The  critical  aspect  of  competence  is  that 
an  individual  must  be  able  to  perform  the  required  tasks  at  an 
acceptable  standard. 

In  February,  1991,  a  national  meeting  sponsored  by  the 
Disabled  Persons  Participation  Program  was  held  in  Ottawa  on  the 
training  of  sign  language  interpreters  with  representation  from 
national  Deaf,  hard  of  hearing  and  deaf-blind  organizations,  in 
addition  to  AVLIC.  At  this  meeting,  the  Canadian  Association  for  the 
Education  of  Sign  Language  Interpreters  (the  Canadian  organization 
of  interpreter  educators)  and  the  Canadian  Association  of  the  Deaf 
were  in  agreement  with  employers  of  interpreters  that  there  is  a 
significant  gap  between  the  skill  level  of  interpreting  program 
graduates  and  the  level  required  to  function  effectively  within  the 
varied  settings  confronting  interpreters. 

In  1990,  the  pass  rate  for  the  performance  segment  of  AVLIC's 
certification  exam  was  21%.  This  low  pass  rate  illustrates  the  lack 
of  skilled  interpreters  in  Canada.  What  skills  are  they  lacking?  How 
can  they  improve?  A  process  for  identifying  the  interpretation 
skills  possessed  by  interpreters  and  comparing  these  to  those  skills 
required  to  perform  competently  is  clearly  needed. 

Requirement  to  Research  Interpreter's  Skills 

The  need  to  empirically  investigate  the  field  of  interpretation 
is  necessary  to  advance  the  interpreting  profession  and  to  provide 
hearing  and  Deaf  consumers  with  competent  ASL/English 
interpreters. 


The  profession  must  investigate  the  elements  of  the 
interpretation  process.  Specifically  what  elements  are 
characterized  by  what  behaviors?  For  example,  to  what  degree  is 
the  source  language,  spoken  English,  equivalent  to  the  interpretation 
in  the  target  language,  ASL?  How  can  a  better  interpretation  be 
provided?  What  are  the  strengths  and  weaknesses  in  an 
interpretation? 

Research  on  the  identification  of  requisite  skills  needed  in  an 
interpretation  will  enhance  interpreter  programs  by  guiding  the 
development  of  curricula  and  materials  to  match  the  identified 
skills.  This  type  of  research  can  also  assist  interpreters  who  are 
preparing  to  take  certifying  examinations  and  are  interested  in 
determining  their  strengths  and  weaknesses  prior  to  taking  the 
examinations. 

Through  research  on  the  skills  required  to  provide  a  competent 
interpretation  and  the  development  of  a  diagnostic  instrument  to 
assess  interpreters,  with  the  ultimate  goal  of  improving  their 
competency  level,  Deaf  people  will  be  assured  quality  access  to  all 
aspects  of  their  daily  lives. 

Delimitations  of  the  Study 

The  goal  of  this  study  was  to  identify  skills  and  develop  an 
assessment  instrument  for  interpreting  from  spoken  English  to  ASL. 

It  did  not  address  interpreting  ASL  to  English.  Nor  did  it  address 
interpreting  other  sign  languages  or  signing  systems  that  represent 
English  such  as  Signing  Exact  English  or  Pidgin  Signed  English. 

Although  many  of  the  identified  skills  were  skills  related  to 
ASL,  the  assessment  instrument  was  designed  to  analyze  these 
skills  only  in  the  context  of  an  interpretation.  That  is,  the 
instrument  was  not  designed  to  assess  ASL  signing  skills,  but  rather 
English-to-ASL  interpreting  skills. 

This  research  addressed  the  issue  of  competence  in  terms  of 
ability  to  do.,  not  in  terms  of  knowledge.  Therefore,  the  question 


was  not  what  one  knows  about  the  languages  used  or  the  topic  being 
interpreted,  but  rather  what  interpreters  do  with  this  information  in 
their  performance. 


CHAPTER  2:  REVIEW  OF  THE  LITERATURE 


Expertise 

Expertise  is  the  ability  to  demonstrate  conceptual  and 
procedural  knowledge  through  rapidly  accessing  mental  functions. 
Over  the  past  25  years,  researchers  have  been  unraveling  the  nature 
of  expertise  (e.g.,  Duda,  1981;  Engle  &  Bukstel,  1978;  Patel, 
Frederiksen,  &  Groen,  1984).  In  a  multitude  of  fields,  scholars  have 
been  conducting  research  to  determine  the  answer  to  the  question: 
"What  is  expertise  and  how  is  it  exhibited?"  (e.g.,  Carter,  Cushing, 
Sabers,  Stein,  &  Berliner,  1988;  Perfetti  &  Lesgold,  1979;  Sloboda, 
1978).  To  date,  the  most  common  way  of  studying  expertise  has 
been  to  compare  and  contrast  the  performance  of  experts  and 
novices  using  the  format  as  a  so-called  ’gold  standard'  (e.g.,  Benner, 
1984). 


Knowledge-lean  and  Knowledge-rich  Expertise 

The  early  studies  of  expertise  were  conducted  during  the 
1960's  and  early  1970's.  Experimental  psychologists  conducted 
research  in  the  area  of  information  processing,  more  specifically 
problem  solving  strategies  (e.g.,  Brown,  1978;  Chase  &  Simon, 

1973a,  1973b;  Newell  &  Simon,  1972).  The  majority  of  these 
experiments  were  in  the  area  of  'knowledge-lean'  tasks,  that  is, 
tasks  which  could  be  learned  in  a  very  short  period  of  time  resulting 
in  competency  (Glaser,  1987,  p.  81).  These  studies  showed  that 
general  cognitive  processes  were  the  same  across  individuals.  Thus, 
each  person  could  achieve  competency  or  expertise  using  general 
knowledge  processes.  No  special  learning  was  required. 

Often  these  studies  used  an  approach  which  defined  the 
baseline  performance,  indicating  a  person's  ability  or  skill  to 
perform  a  specific  task  under  study  prior  to  any  experience.  This 
baseline  was  then  compared  to  the  outcome  of  the  experiment  in 


terms  of  competency  achieved.  Often  this  was  a  one-time 
measurement  and  was  done  under  standardized  conditions,  including 
the  instructions  dealing  with  the  tasks  involved  and  the  manner  in 
which  to  respond. 

The  studies  were  enlightening;  however,  the  research  did  not 
address  learning  and  thinking  which  occur  over  an  extensive  period 
of  time  and  with  a  great  deal  of  experience.  These  studies  lacked 
research  into  the  deep  structure  of  domain-specific  knowledge,  that 
is  'knowledge-rich'  tasks,  tasks  that  require  an  abundance  of 
learning  over  time  with  significant  amounts  of  experiences  related 
to  a  particular  area  (Glaser,  1987,  p.  82). 

In  recent  years,  researchers  such  as  Anderson  (1981,  1982), 
Baltes,  Dittmann-Kohli,  and  Dixon  (1984),  and  Chi,  Glaser,  and  Farr 
(1988)  have  focused  much  more  on  studies  of  expertise,  particularly 
in  the  area  of  knowledge-rich  tasks.  Experts  such  as  master  chess 
players,  nurses,  teachers,  and  radiologists  have  learned  their  skills 
over  a  much  more  extended  period  of  time  than  the  knowledge-lean 
skills  studied  in  the  1960's  and  1970's.  Even  novices  within  each  of 
these  areas  have  spent  more  time  in  learning  than  the  prior 
experimental  studies  in  experimental  psychology  (Glaser,  1987). 

Models  of  adult  cognition  and  intellectual  development  have 
tried  to  deal  with  the  interrelatedness  of  experiences  and  cognitive 
processes  as  it  relates  to  specified  knowledge  areas  (Baltes  et  al., 
1984;  Baltes  &  Kliegl,  1986;  Charness,  1981;  Denney,  1984;  Hoyer, 
1985;  Labouvie-Vief,  1982,  1985;  Salthouse,  1985).  More 
specifically,  cognitive  theories  have  tried  to  examine  this  issue  in 
relation  to  differences  between  experts  and  novices  (Brown,  1982; 
Chi,  Glaser,  &  Rees,  1982).  Thus,  levels  of  expertise  were  studied. 

Differences  Between  Novices  and  Experts 

Every  occupation,  such  as  chess  players  (Chase  &  Simon, 

1973a;  de  Groot,  1965;  Patel  et  al.,  1984),  bridge  players  (Engle  & 
Bukstel,  1978),  teachers  (Carter  &  Berliner,  1987),  and  nurses 


(Benner,  1984)  have  individuals  who  range  in  skill  and  knowledge 
from  novices  to  experts.  There  are  five  levels  of  skills  cited  in  the 
literature.  These  are  novice,  advanced  beginning,  competence, 
proficiency,  and  expertise  (Dreyfus,  Dreyfus,  &  Athanasiou,  1986). 
Each  level  is  more  advanced  than  the  previous  level. 

The  novice  is  characterized  as  having  literal  knowledge  about 
objects  and  events.  The  depth  of  representational  knowledge  is 
limited  (Anderson,  1981;  Calderhead,  1983).  In  contrast,  the  expert 
has  developed  the  ability  to  use  abstraction  and  make  inferences 
about  principles  involved  in  the  situation. 

Chi,  Feltovich,  and  Glaser  (1981)  studied  problems  in 
mechanics.  They  found  novices  classify  problems  according  to  the 
physical  properties  of  situations  such  as  the  valve  problem  or  strut 
problem.  Experts  have  the  ability  to  categorize  at  a  more  complex 
schematic  level  by  applying  principles  of  physics  to  the  specific 
problem  noted  and  the  impact  of  this  principle  on  the  outcome  of  the 
solution. 

Novices,  according  to  Dreyfus  et  al.  (1986)  and  others  (e.g., 
Benner,  1984),  have  the  ability  to  recognize  objective  facts 
relatively  easily  but  lack  contextualism  when  dealing  with  various 
situations  and  therefore  rely  on  rote  answers.  Novices  are 
followers  of  rules  and  it  is  these  rules  which  govern  their  actions. 

Dreyfus  et  al.  (1986)  and  Benner  (1984)  go  on  to  say  that  as 
novices  progress  to  a  higher  level  of  skill  they  develop  experience  in 
dealing  with  real  situations.  Advanced  beginners  have  the  ability  to 
contextualize  the  situation  more  broadly  than  novices.  Advanced 
beginners  see  the  situation  more  as  a  whole,  rather  than  just 
unrelated  and  isolated  parts. 

These  same  researchers  consider  the  skill  level  of  competence 
to  be  half-way  between  novice  and  expertise.  Competence  is 
characterized  by  the  ability  to  view  the  situation  as  a  set  of  facts 
rather  than  as  individual  components  that  make  up  the  situation. 

The  person  who  is  competent  also  has  the  ability  to  rule  out  the 
unimportant  factors  and  deal  with  only  the  salient  features  of  the 


situation.  The  competent  person  is  not  looking  at  it  from  the 
outside  but  is  engrossed  in  the  inner  workings  of  the  situation. 

Proficient  individuals  not  only  see  the  situation  as  a  whole, 
but  also  rely  on  experience  to  make  decisions  about  the  best  course 
of  action  to  take  in  a  given  situation.  Their  experience  permits  them 
to  reflect  on  the  long  term  implications  of  the  array  of  possible 
decisions  that  exists. 

The  highest  level  of  skill,  that  of  expertise,  is  characterized 
by  rapid,  fluid,  and  involved  behavior  that  is  quite  different  from  the 
slow,  uninvolved  mental  processes  used  in  the  problem  solving 
process  seen  at  the  earlier  stages  of  skill  development  (Dreyfus  et 
al.,  1986).  Thought  is  not  given  to  how  something  should  be  done, 
but  attention  is  given  to  the  intuitive  decisions  that  come  rapidly 
and  successively  while  the  person  is  involved  in  the  situation.  A 
gestalt  viewpoint  is  used  while  features  related  to  the  situation  or 
task  come  to  the  foreground  and  other,  more  irrelevant  information 
fades  to  the  background.  Practice  provides  experience,  a  key 
element  in  addition  to  actually  developing  the  skill. 

Experts  are  better  able  to  remember  specific  meaningful 
information  such  as  facts  and  patterns  in  a  specific  area  of 
expertise.  In  1965,  de  Groot  demonstrated  this  among  chess  players. 
His  research  was  later  replicated  in  several  studies  of  chess 
players,  for  example  Chase  and  Simon  (1973b)  and  Patel, 

Frederiksen,  and  Groen  (1984). 

Similarly,  Larkin,  McDermott,  Simon,  and  Simon  (1980)  found 
that  the  most  striking  difference  between  expert  and  novice 
performance  in  solving  physics  problems  is  that  "the  expert  knows  a 
great  many  things  the  novice  does  not  know  and  can  rapidly  evoke 
the  particular  items  relevant  to  the  problem  at  hand"  (p.  1136). 

Engle  and  Bukstel  (1978)  concluded  their  study  by  stating,  "bridge 
players  with  'supranormal'  memory  are  able  to  use  their  prior 
experience  to  configure  and  chunk  information  in  more  efficient 
ways  than  players  of  less  expertise"  (p.  673).  The  ability  to 
perceive  information  in  chunks  in  order  to  determine  meaning,  which 


is  often  said  to  be  nearing  'intuition1,  is  essential  to  the  functioning 
of  experts  (Chi  et  al.,  1982;  Lesgold,  1984). 

Experts  use  the  ability  to  monitor  their  own  behavior  and 
constantly  adjust  to  fit  the  demands  of  the  situation  because  they 
have  developed  automaticity  in  performing  the  "basic  operations"  of 
the  task.  In  this  way  their  working  memory  is  available  for  higher 
level  conscious  processing  (Perfetti  &  Lesgold,  1979).  More 
specifically,  a  key  feature  of  behavior  exhibited  by  experts  is  the 
ability  to  use  self-regulating  processes  as  a  result  of  experience 
(Brown,  1978;  Gitomer  &  Glaser,  1985).  These  cognitive  skills  are 
manifested  by  proficiency  in  techniques  of  solution  monitoring,  by 
the  allocation  of  attention,  and  by  sensitivity  to  informational 
feedback. 

In  the  field  of  interpretation,  interpreters  must  determine  the 
best  interpretation  based  on  the  contextual  cues  of  the  situation. 

For  example,  a  specific  English  phrase  is  spoken  but  the  meaning  is 
ambiguous;  "As  you  pass  out  please  hand  in  your  papers."  Does  the 
phrase  pass  out  mean  to  faint?  Is  the  instructor  being  sarcastic  or 
humorous?  Does  the  instructor  simply  mean,  as  you  leave,  please 
hand  in  your  papers?  The  interpreter  must  make  a  decision  within 
seconds  of  the  utterance.  Based  on  the  interpreter's  experience  and 
the  familiarity  with  the  speaker  and  the  setting,  the  interpreter 
will  make  a  decision.  Whether  it  is  the  right  one  depends  on  the 
interpreter's  expertise  to  evaluate  the  setting  and  the  utterance, 
using  all  of  the  available  cues.  In  ASL/English  interpreting  this  is 
demonstrated  when  the  Deaf  consumer  shows  a  quizzical  expression 
indicating  a  lack  of  comprehension.  Within  seconds,  the  interpreter 
will  evaluate  the  problem.  The  interpreter  may  identify 
immediately  that  the  interpreted  message  contained  an  incorrect 
sign  or  the  interpreter  neglected  to  add  a  negation,  thereby 
mistakenly  making  the  statement  affirmative,  which  did  not  make 
sense  to  the  Deaf  person.  After  determining  what  interpretation 
error  occurred,  the  interpreter  will  make  the  necessary  adjustment 
to  rectify  the  error.  In  this  example,  the  interpreter  is  exhibiting 
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the  ability  to  respond  to  informational  feedback  by  monitoring  the 
solution  and  mentally  replaying  a  portion  of  the  confusing 
interpretation.  At  the  same  time  the  interpreter  must  maintain 
concentration  not  only  to  correct  the  error,  but  also  to  continue 
attending  to  and  interpreting  the  speaker's  ongoing  message. 

Weaknesses  In  the  Novice  and  Expert  Literature 

The  research  related  to  expertise  has  several  limitations.  One 
of  the  most  obvious  issues  is  the  limited  number  of  experts  in  the 
specific  domain  being  studied  (Kliegl  &  Baltes,  1987).  Thus,  the 
pool  of  possible  research  subjects  is  restricted.  The  number  of 
participants  in  a  study  tends  to  be  small  (e.g.,  Carter,  Sabers, 
Cushing,  Pinnegar,  &  Berlinger,  1987).  In  Chi,  Glaser,  and  Rees' 

(1982)  study  comparing  expert  and  novice  performance  in  solving 
problems  in  elementary  physics,  the  researchers  used  only  one 
expert  and  one  novice.  Generalizability  of  the  findings  becomes  an 
obvious  issue. 

The  issue  is  further  confounded  when  peers  evaluate  the  few 
qualified  experts.  Similar  difficulties  arise  when  the  selection 
procedures  are  primarily  based  on  the  opinions  and  observations  of 
co-workers  and  superiors  (e.g.,  Gitomer  &  Glaser,  1985;  Hoyer,  1985; 
Labouvie-Vief,  1985;  Lesgold,  McDermott,  Simon,  &  Simon,  1980). 
The  experts  are  often  the  perceived  experts.  The  perceived  experts 
then  become  the  raters  and  evaluators.  Thus,  everything  is  in 
relation  to  them.  The  characteristics  identified  are  supposedly 
those  of  experts.  However,  do  these  experts  have  other 
characteristics  in  addition  to  the  expertise  itself  which  encouraged 
people  to  support  them  as  experts?  For  example,  would  experts  be 
acknowledged  as  experts  if  they  had  poor  interpersonal  skills? 

Acquisition  of  knowledge-rich  expertise  requires  many  years 
of  practice  in  the  relevant  activity.  Research  on  the  natural 
development  of  expertise  over  time  is  severely  limited.  Thus, 
questions  to  be  answered  are:  How  is  the  level  of  expertise 


decided?  What  competencies  or  skills  must  people  possess  to  be 
classified  as  experts?  How  does  one  move  from  the  novice  to  the 
expert  level? 

In  many  instances,  issues  arise  such  as  the  amount  of  time 
required  to  master  a  skill.  If  one  wants  to  be  an  expert  playing  the 
guitar  in  a  matter  of  months,  this  is  not  possible.  How  is  this 
handled?  Must  one  go  through  all  the  levels  before  becoming  an 
expert?  How  much  time  does  it  take?  Is  there  an  average  amount  of 
time  or  does  it  depend  on  the  individuals  involved  or  the  specific 
skills  being  learned? 

The  role  of  experience  or,  perhaps,  more  accurately  the  role  of 
reflected-on  experience  in  the  development  of  expertise  is 
acknowledged,  but  is  not  yet  well  understood.  Comparative  studies 
of  skills  and  knowledge  at  various  levels  of  experience  and  expertise 
will  provide  greater  understanding. 

ASL/English  Interpreting  Education 
Post-Secondary  Programs 

As  stated  above,  formal  education  of  ASL/English  interpreters 
began  in  the  1970's  in  the  United  States  (Neumann  Solow,  1981).  In 
Canada,  comprehensive  education  in  interpreting  did  not  begin  until 
the  1980's  (Schein  et  al.,  1989).  The  profession  of  ASL/English 
interpreting  is  relatively  new  in  North  America.  Therefore,  research 
related  to  the  expertise  of  ASL/English  interpreters  is  relatively 
undeveloped. 

One  of  the  most  recent  Canadian  reports  on  interpreting,  as 
well  as  other  related  issues,  was  conducted  in  Ontario.  This 
provincial  review  states  that  due  to  the  "open  door"  policies  of 
interpreting  programs  in  Ontario,  graduates  are  not  prepared  to 
enter  the  interpreting  profession  upon  graduation  (Ministry  of 
Colleges  and  Universities,  1992). 
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At  the  federal  level  in  Canada,  the  Disabled  Persons 
Participation  Program  (DPPP),  through  meetings  with 
representatives  from  a  range  of  stakeholders,  including  interpreter 
educators,  program  administrators  of  post-secondary  interpreting 
programs,  and  deaf  consumers,  stated  that  many  graduates  of 
Canadian  interpreting  programs  were  unable  to  adequately  function 
as  interpreters  because  of  a  lack  of  skill  upon  graduation. 

Because  of  the  infancy  of  formal  education  in  ASL/English 
interpreting,  it  is  not  surprising  that  the  criteria  for  graduating 
from  these  programs  is  not  sufficient  for  graduates  to  work 
competently  as  interpreters. 


Curricula 

According  to  interpreter  educators  such  as  Arjona  (1984), 
curricula  used  in  interpreter  programs,  both  spoken  and  signed, 
across  North  America  are  based  on  the  experiences,  opinions,  and 
philosophies  of  the  instructors  and  the  college  and  university 
departments. 

McKee  (1980)  stated  that  much  of  the  literature  on  curricula 
related  to  the  teaching  of  interpreters  is  directly  related  to  the 
common  settings  in  which  interpreters  work.  This  is  still  prevalent 
in  some  of  the  most  recent  curricula  available  in  North  America.  For 
example,  the  University  of  New  Brunswick  Sign  Language  Interpreter 
Training  Curriculum  (Cokely,  1988)  includes  specific  settings,  such 
as  educational  and  medical,  as  well  as  the  number  of  individuals 
involved  in  the  setting  requiring  an  interpreter.  The  curriculum 
covers  whether  the  interpreter  will  interpret  for  two  people  or  a 
large  group  of  people  interacting.  Undoubtedly,  these  are  necessary 
components  of  interpreting  but  the  primary  question  remains;  what 
underlying  skills  and  behaviors  are  necessary  to  interpret  in  these 
settings? 

Another  area  included  in  texts  used  in  interpreting  programs 
(e.g.,  Frishberg,  1986)  and,  more  specifically,  in  curricula  (e.g., 
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University  of  New  Brunswick  Curriculum)  is  information  on  how  to 
work  with  Deaf  consumers,  such  as  Deaf-blind,  oral,  or  individuals 
with  minimal  language  skills.  Indeed,  this  is  an  integral  part  of  the 
curriculum  and  of  interpreters'  experience,  but  first  interpreters 
must  be  able  to  interpret  well  before  they  can  competently  work 
with  various  consumers.  It  does  not  matter  how  well  interpreters 
handle  the  settings  or  the  consumers  in  the  settings  if  they  do  not 
have  the  skills  to  handle  the  interpretation  task. 

Interpreters  must  acquire  a  solid  foundation  of  interpreting 
skills  before  they  can  be  educated  to  work  in  specific  settings  and 
with  specific  consumers.  Without  attention  to  the  underlying  skills, 
interpreters  may  never  acquire  the  level  of  expertise  necessary  to 
interpret  well.  They  may  acquire  skills  for  working  in  specific 
settings  and  with  specific  consumers,  but  this  does  not  necessarily 
mean  interpreters  will  possess  the  interpreting  skills  required  to 
render  a  competent  interpretation. 

Since  the  1988  University  of  New  Brunswick  curriculum,  a 
more  recent  four  volume  curriculum,  developed  from  1987-1990  and 
titled  Interpreter  Training  Course  Curriculum  Guides  (Taff-Watson  & 
Northup,  1987-1990)  at  the  University  of  Arkansas  at  Little  Rock, 
has  been  published.  Even  though  the  title  of  this  curriculum 
indicates  it  is  for  interpreter  training,  its  primary  focus  is  not  on 
acquiring  skills  and  knowledge  related  to  interpreting,  but  instead 
on  acquiring  skills  and  knowledge  related  to  ASL  and  working  with 
various  consumers. 

Volume  I  (1987)  contains  five  course  curriculum  guides.  The 
focus  is  primarily  ASL  language  development  and  methods  for 
communicating  with  deaf  people.  For  example,  Manually  Coded 
English  and  speechreading,  as  well  as  working  with  specific 
consumers  who  are  oral  deaf  people,  not  signers,  are  covered.  No 
courses  in  Volume  1  cover  ASL/English  interpreting. 

Volume  2  (1988)  also  includes  five  course  curriculum  guides. 
Three  courses  include  ASL,  fingerspelling,  and  signing  Manually 
Coded  English.  The  remaining  two  courses  are  dedicated  to 


ASL/English  interpreting,  one  related  to  theory  and  the  other  related 
to  interpreting  skill  development.  Thus,  only  two  out  of  the  five 
curriculum  guides  focus  on  interpreting. 

The  curriculum  designed  to  develop  interpreting  skills  includes 
a  student  evaluation  form  that  has  a  list  of  criteria  for  rating 
interpreting  students  on  a  5-point  scale.  Under  the  portion 

'Interpreting  Factors:  English  to  ASL',  10  factors  are  given.  For 

example,  'sign  utilization',  'message  conveyed',  fluency',  and  'use  of 
space'  are  listed.  Definitions  are  not  provided  with  this  listing. 

Thus,  as  Arjona  (1984)  mentioned,  the  understanding  of  the 
interpreting  factors  and  the  rating  is  based  on  individual 
instructors'  experience  and  knowledge. 

Volume  3  (1989)  contains  curriculum  guides  for  interpreting 
for  people  who  are  deaf-blind  and  curriculum  on  conceptual  sign 
language.  In  addition,  it  addresses  voice  production  for  interpreters 
and  ASL-to-English  interpreting.  The  latter  two  curricula 

specifically  address  interpreting  skills.  The  skills  related  to 

speaking  clearly  are  covered  in  depth  in  the  voice  production 
curriculum.  These  skills  are  required  to  render  a  clearly  articulated 
message  in  English  regardless  of  the  consumers  or  the  settings.  In 
addition,  much  of  the  content  of  the  ASL-to-English  curriculum  is 
appropriately  related  to  voice  production.  Detailed  sample 
checklists  are  provided  and  in  which  speech  qualities  such  as 
'inhalation  is  silent,  quick  and  unobtrusive'  and  the  speech  is  'clear, 
distinct  and  accurate'  are  listed.  Pre-interpreting  activities  for 
interpreting  from  ASL  to  English  such  as  abstracting  and 
paraphrasing  are  also  described.  Very  little  is  devoted  to  the  actual 
skills  of  interpreting  ASL  to  English,  that  is,  comprehending  ASL  and 
rendering  it  into  its  equivalent  English  meaning. 

Five  workshop  guides,  not  courses,  are  included  in  the  final 
volume,  Volume  4  (1990).  One  workshop  is  devoted  to  interpreting 
for  persons  with  minimal  language  skills  which  includes  specialized 
interpreting  skills  required  to  work  with  certain  consumers.  Two 
workshops  cover  interpreting  in  specific  settings:  interpreting  in 
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business  and  industry  and  in  mental  health.  The  remaining  two 
chapters  are  'Visualization  and  Imagery  for  Interpreters',  and 
'Teaching  Children  Who  are  Deaf  to  Use  Interpreters'.  The  former 
workshop  includes  developing  the  mental  skills  necessary  for 
interpreting,  but  the  latter  again  specifically  addresses  settings  and 
consumers. 


Testing  and  Certification 

Experts  are  commonly  defined  as  persons  whose  performance 
on  a  criterion  task  is  far  superior  to  that  of  normal  persons  (e.g. ,  Chi 
et  al. ,  1982;  Emmer,  Evertson,  &  Anderson,  1980;  Evertson  &  Emmer, 
1982).  Qualitative  measures  differentiate  novices  and  experts  with 
the  experts'  performance  being  'better',  thereby  outperforming  the 
novices  (e.g.,  Carter  et  al.,  1987).  What  does  'better'  mean?  Are  the 
measurements  used  adequate  to  determine  the  superior  performance 
identified? 

What  are  other  ways  of  determining  levels  of  skills  and 
competencies?  One  way  of  assessing  skills  is  through  examination. 
Once  people  have  completed  the  necessary  requirements,  such  as 
education  and/or  experience,  then  often  they  must  take  certification 
examinations  at  provincial,  state,  and/or  national  levels. 

In  the  early  1900's,  only  a  handful  of  certified  or  licensed 
occupations  and  professions  existed  in  the  United  States,  compared 
to  the  early  1980's  when  there  were  about  800  (Shimburg,  1985). 
Public,  government,  and  professional  organizations  have  recognized 
the  importance  of  competency  and  have  demanded  services  be 
provided  by  professionals  who  have  achieved  recognized  and 
independently  assessed  levels  of  competency. 

According  to  Berk  (1986)  examinations  used  for  the 
certification  of  various  professions  and  occupations  are  set  by 
members  of  that  particular  profession  or  occupation.  This  is  also 
true  in  the  area  of  certifying  ASL/English  interpreters.  Both  AVLIC 
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and  RID  use  experts  to  make  judgements  as  to  the  competency  of 
interpreters. 

Loveland  (1976)  stated  that  the  determination  of  what 
constitutes  competence  is  the  most  critical  factor  in  test 
development.  Any  profession  must  begin  with  the  identification  of 
the  necessary  skills  that  comprise  the  performance  domain. 

ASL/English  Interpreting  Assessment 
Interpreters1  and  Deaf  People's  Intuition 

Interpreters  and  Deaf  people  based  on  their  experiences  and 
exposure  to  various  competency  levels  of  interpreters,  often  have  an 
intuitive  impression  about  who  are  good  interpreters  and  who  are 
poor  interpreters.  The  question  then  becomes  how  is  this  known? 
What  are  the  criteria  for  this  intuitive  viewpoint? 

Experienced  interpreters'  and  Deaf  consumers'  intuitive 
reactions  toward  an  interpretation  do  have  merit  (Per-Lee,  1981), 
but  what  is  this  merit?  How  can  interpreters'  skill  level  be 
improved?  Most  important  of  all,  what  is  considered  competent  and 
how  is  this  measured?  What  criteria  are  used  when  determining 
competency? 

RID  National  Certification 


In  the  United  States,  the  Registry  of  Interpreters  for  the  Deaf 
(RID)  first  offered  national  evaluations  for  certification  of 
English/ASL  interpreters  in  1972  (Frishberg,  1990).  Interpreters 
who  were  successful  on  the  entire  evaluation  were  granted  the 
Comprehensive  Skills  Certificate  (CSC). 

The  CSC  evaluation  was  weighted  based  on  skills  and  the 
importance  of  those  skills.  When  assessing  English-to-ASL 
interpretations,  the  skills  evaluated  were  stated  in  general  terms, 
for  example,  'non-manual  behaviors',  'use  of  space',  'speed/time  lag', 
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and  'message  conveyed'.  A  total  of  10  factors  were  assessed  on  a  5- 
point  rating  scale  (e.g.,  from  inappropriate  to  appropriate  or  skewed 
to  accurate).  A  panel  of  Deaf  and  hearing  raters  independently  rated 
the  live  performances. 

About  10  years  after  the  evaluation  of  interpreters  began, 
Culton  (1981)  examined  the  validity  and  reliability  of  the  CSC.  He 
stated,  "the  results  of  this  study  cast  doubts  on  the  reliability  of 
scoring  methods  on  the  Comprehensive  Skills  Certificate  evaluation, 
as  the  scoring  patterns  of  various  evaluating  panels  appeared  to 
differ  significantly.  The  California  evaluation  panels  used  for  this 
study  produced  scoring  patterns  at  variance  with  the  general  scoring 
patterns  of  over  fifty  panels  throughout  North  America  in  four  out  of 
five  sections  of  the  evaluation  studied"  (p.  52). 

More  recently,  in  1987,  the  evaluation  system  was  changed 
from  a  criterion-referenced  to  a  proficiency-based  system,  again 
using  trained  expert  raters  (Fant,  1990).  Certified  members,  holding 
a  CSC,  viewed  video  tapes  at  the  RID  1987  biennial  conference  held 
in  St.  Paul,  Minnesota,  and  determined  the  cut  off  scores  for  passing. 
The  performance  domain  was  determined  to  include  not  only 
interpreting,  but  also  languages,  that  is,  English  and  ASL. 

The  impact  on  interpreters  taking  the  test  is  that  when  they  receive 
their  results  they  are  informed  of  their  pass  or  fail  mark.  If  the 
interpreters  pass,  they  are  told  only  that  they  passed.  Nothing  is 
mentioned  as  to  how  well  they  did  or  areas  they  may  need  to 
improve.  It  is  not  diagnostic.  If  they  fail,  no  indication  is  made  as 
to  what  portions  they  failed  or  how  far  they  fell  short  of  the 
minimum  criteria. 

AVLIC  National  Certification 

The  Association  of  Visual  Language  Interpreters  of  Canada 
(AVLIC)  began  certifying  interpreters  in  1990  (Janzen  &  Demers, 
1990).  Similar  to  the  revised  RID  certificate,  the  Test  of 
Interpretation  (TOI)  is  a  proficiency-based  performance  test. 
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Experts  determined  the  minimum  scores  for  passing.  The 
performance  domain,  like  the  RID's,  was  determined  to  include 
competency  in  not  only  the  message  equivalency  of  the 
interpretation,  but  also  English  and  ASL. 

Successful  interpreters  must  perform  at  an  acceptable  level  at 
each  domain  to  be  granted  certification.  The  raters  received 
training  and  were  said  to  be  consistent  with  each  other  in  their 
decisions  (Evans,  1991). 

Again  the  questions  remain:  What  skills  do  expert  interpreters 
demonstrate?  If  interpreters  fail,  what  do  they  need  to  achieve 
before  they  are  able  to  pass?  What  skills  do  they  already  possess? 

Summary 

Deaf  and  hearing  consumers  and  employers  of  interpreters  are 
in  agreement  that  many  graduates  of  ASL/English  interpreting 
programs  lack  the  necessary  skills  to  interpret  competently  upon 
graduation.  This  is  confirmed  by  the  fact  that  interpreters  who  took 
the  AVLIC  performance  exam  in  1990  had  a  failure  rate  of  79%. 
According  to  the  RID  Views  (November,  1992),  data  related  to  the 
number  of  interpreters  who  passed  or  failed  the  performance 
examinations  were  not  kept  for  the  fiscal  years  1988-1989  and 
1989-1990.  However,  interpreters  who  took  the  RID  performance 
exam,  for  ASL/English  interpretation  in  fiscal  year  1990-1991  and 
received  their  results,  had  a  failure  rate  of  approximately  54%. 
Interpreters  who  took  the  exam  in  1991-1992,  and  received  their 
results,  had  a  failure  rate  of  approximately  27%. 

A  proficiency-based  evaluation  indicating  a  pass  or  fail  has  its 
purpose.  The  goal  of  certification  is  competency  and  not  diagnostic. 
However,  to  provide  the  test  taker  with  only  a  pass  or  fail  mark  does 
not  provide  information  as  to  what  the  level  of  competency  was  and 
what  it  was  not.  What  were  the  strengths  and  weaknesses?  What 
should  be  done  to  improve?  Even  with  passing  marks  of  'competent', 
interpreters  may  still  have  areas  of  weaknesses.  Granted,  this  is 
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not  the  purpose  of  many  certification  examinations.  However,  these 
areas  of  weakness  could  be  addressed  if  interpreters  were  given  the 
necessary  information. 

One  method  of  providing  information  to  interpreters  is  to 
provide  diagnostic  assessments  of  their  interpretations.  With  the 
results  of  these  assessments,  interpreters  can  use  the  information 
to  progress  to  a  higher  level  of  expertise. 

If  this  is  to  occur,  first  specific  skills  must  be  identified  and 
then,  a  method  for  assessing  the  skills  must  be  developed. 

Following  these  two  steps,  curricula  must  be  developed  that  not  only 
list  requisite  skills  of  expert  interpreters,  but  defines  them  so  that 
instructors  and  the  departments  of  colleges  and  universities  have 
more  to  rely  on  than  simply  their  own  expertise.  Although  expertise 
is  helpful,  experts  have  varying  degrees  of  expertise  and  may  have 
different  philosophies  and  opinions.  Hence,  the  skills  of  expert 
interpreters  must  be  more  objectively  determined  and  defined.  With 
this  in  place,  curricula  can  be  enhanced,  thus  enhancing  interpreting 
programs  and  graduates'  skill  level  upon  graduation. 

Further  research  is  needed  in  the  areas  of  novice  and  expert 
levels  of  competency.  All  disciplines  without  exception  can  benefit 
from  this  type  of  research.  How  can  people  improve?  How  is 
expertise  defined  within  a  profession,  specifically  ASL/English 
interpreting,  so  that  it  assists  interpreters  in  their  development 
from  novice  to  expert  levels  of  expertise? 

Both  curricula  and  national  certification  examinations  lack  the 
detailed  identification  of  the  necessary  skills  to  perform  an  expert 
interpretation.  Thus,  this  research  focused  on  the  skills  required  to 
render  an  expert  interpretation,  specifically,  interpreting  from 
English  to  ASL.  The  current  research  included  the  identification  and 
definition  of  skills,  and  the  development  of  a  diagnostic  assessment 
instrument  to  assess  interpreters  at  various  skill  levels  and 
experiences.  The  methods  used  and  the  findings  are  described  in  the 
balance  of  this  thesis. 
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CHAPTER  3:  INSTRUMENT  DEVELOPMENT  AND  EXPERT  PANEL  REVIEW 

Overview 

The  goal  of  the  present  research  was  to  develop  a  practical  and 
useful  diagnostic  instrument  for  assessing  interpretations  from 
spoken  English  to  its  equivalent  in  American  Sign  Language  (ASL). 
The  instrument  was  developed  by  identifying  skills  generally  seen  in 
expert  English-to-ASL  interpretations.  Identification  was 
accomplished  by  reviewing  the  related  literature  and  conducting 
observational  analysis  of  various  interpreters  with  different  skill 
levels  and  experience. 

Once  the  skills  were  identified  they  were  grouped  by  the 
researcher  into  eight  categories  or  Major  Features:  1)  Numbers,  2) 
Fingerspelling,  3)  Classifiers/SASSes,  4)  Vocabulary,  5)  Structuring 
Space,  6)  Grammar,  7)  Interpreting,  and  8)  Composure  and 
Appearance.  A  ninth  category,  Rater's  Judgements,  was  also  added 
to  provide  the  rater  with  an  opportunity  to  make  judgements  and 
provide  further  diagnostic  assessment  on  the  overall  performance. 

Following  an  error  analysis  of  interpretations,  lists  of 
relevant  and  representative  errors  that  might  be  made  by  an 
interpreter  were  developed  for  each  skill  within  each  Major  Feature. 

A  systematic  approach  to  test  construction  as  outlined  by 
Crocker  and  Algina  (1986,  pp.  66-84)  was  followed  to  develop  the 
instrument.  The  first  seven  of  the  ten  steps  outlined  by  Crocker  and 
Algina  were  used.  These  were:  1)  determine  the  purposes  for  which 
the  assessment  scores  derived  from  the  instrument  would  be  used, 

2)  identify  and  describe  the  behaviors  indicative  of  good 
interpretation  practice,  3)  create  the  test  specifications  and  the 
method  of  recording  observed  behaviors,  4)  construct  the  initial  pool 
of  items,  5)  review  of  the  instrument  by  a  panel  of  interpretation 
experts,  6)  field  test  the  instrument  using  interpreter  educators, 
and  7)  analyze  the  data. 
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The  work  and  the  results  corresponding  to  the  first  five  steps 
are  described  in  the  balance  of  this  chapter.  The  work  and  results 
associated  with  steps  six  and  seven  are  described  in  Chapters  4  and 
5,  respectively. 


Instrument  Development 

Step  1:  Determine  the  Purposes  For  Which  The  Assessment  Scores 

Derived  From  the  Instrument  Would  Be  Used 

The  purpose  of  the  assessment  scores  derived  from  the 
instrument  was  to  provide  interpreters  at  various  skill  levels  and 
experience  with  a  detailed  diagnostic  assessment  of  their  English- 
to-ASL  interpretation  skills. 

Because  the  intent  was  to  assess  interpretation  skills,  the 
assessment  of  transliterating  skills  using  invented  sign  systems, 
such  as  Signing  Exact  English  or  Signing  Essential  English,  was 
excluded.  Transliteration  occurs  when  spoken  English  is  conveyed 
manually  in  a  signed  form  of  English.  This  method  of  relaying 
information  entails  the  use  of  two  modes,  aural  and  manual,  but  does 
not  include  the  use  of  two  languages.  It  deals  only  with  one 
language,  English.  Therefore,  assessing  transliteration  or  invented 
sign  systems  was  not  included  in  the  instrument. 

The  instrument  was  designed  to  be  useful  and  practical  for 
interpreters  and  raters.  This  was  accomplished  in  two  ways.  First, 
current  terminology  commonly  understood  within  the  interpreting 
profession  was  employed  in  the  wording  of  the  instrument.  The 
wording  used  throughout  the  instrument  is  what  appears  in  much  of 
the  available  literature.  Second,  attempts  were  made  to  keep  the 
instrument  a  manageable  length  so  that  it  would  not  require  an 
excessive  amount  of  time  to  assess  an  English-to-ASL 
interpretation,  while  at  the  same  time  maintaining  a  device  of 
sufficient  comprehensiveness  to  ensure  its  practical  utility. 


Step  2:  Identify  and  Describe  the  Behaviors  Indicative  of  Good 

Interpretation  Practice 


32 


Information  Sources 

Having  determined  the  purpose  of  the  instrument,  the  domain 
was  defined  as  English-to-ASL  interpreting.  Because  the  instrument 
was  to  provide  interpreters  with  a  detailed  assessment  of  their 
English-to-ASL  interpretations,  it  was  necessary  to  clearly  define 
the  relevant  skills  required  to  provide  an  expert  interpretation  from 
English  to  ASL. 

Information  was  collected  from  a  variety  of  sources  to  define 
the  domain  of  English-to-ASL  interpreting.  First,  the  literature  was 
reviewed  to  assist  in  identifying  the  range  of  skills  demonstrated  by 
novices  and  experts  (interpreting  students  to  expert  or  certified 
interpreters).  Altogether  72  sources,  which  are  listed  in  Appendix 
A,  were  reviewed. 

Another  source  of  information  included  observations  of 
interpreters  who  were  viewed  both  directly  and  on  video  tape.  All 
observations  were  conducted  by  the  researcher  and  included 
documenting  the  successful  and  less  than  successful  elements  of  the 
interpretations.  The  direct  observations  were  done  live  with  no 
video  tape  available  of  the  work.  In  contrast,  the  video  taped 
interpretations  were  reviewed  several  times. 

There  were  439  direct  observations  and  128  video  taped 
observations.  The  direct  observations  were  of  interpreters  with  a 
variety  of  skill  levels  and  experience.  These  included  a  range  from 
novice  to  expert.  The  novices  included  students  in  an  interpreting 
program  or  graduates  of  these  programs  with  less  than  two  years  of 
interpreting  experience.  The  expert  interpreters  had  five  or  more 
years  of  interpreting  experience  and/or  held  national  certification 
in  interpreting  from  either  the  Association  of  Visual  Language 
Interpreters  of  Canada,  the  Registry  of  Interpreters  for  the  Deaf,  or 
both. 
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The  direct  observations  of  interpreters  were  conducted  in  a 
number  of  settings  including  employment  (n=111),  educational 
(n=95),  platform  (n=78),  meetings  (n=70),  workshops  (n=52),  and 
medical  assignments  (n=33). 

The  video  taped  observations  included  both  novice  (N=85)  and 
expert  (N=43)  interpreters.  Just  over  half  of  the  novice  interpreters 
(N=44)  were  students  who  attended  the  Interpreter  Training  Program 
at  Grant  MacEwan  Community  College  in  Edmonton,  Alberta  during 
the  period  1984  to  1989.  The  remaining  novices  (N=41)  were 
interpreters  from  Canada  and  the  United  States  with  less  than  two 
years  of  experience.  The  expert  video  tape  samples  were  of 
interpreters  who  provided  modelled  interpretations  for  interpreting 
programs  at  post-secondary  institutions.  Also,  commercially  sold 
video  tapes  were  used,  as  well  as  video  tapes  made  from  television 
programs  where  interpretations  were  provided.  A  list  of  the 
modelled  interpretation  video  tapes  used  in  this  part  of  the  research 
is  provided  in  Appendix  B. 

Task  Analysis 

Using  the  information  gleaned  from  the  literature  and  the 
observations  of  interpreters,  a  task  analysis  was  conducted  to 
define  the  assessment  domain  in  terms  of  the  skills  required  to 
provide  equivalency  of  meaning  from  the  source  language  of  English 
to  the  target  language  of  ASL.  The  analysis  revealed  that 
interpreters  who  provided  more  expert  interpretations  possessed 
both  excellent  interpreting  skills  and  ASL  language  skills.  A 
common  characteristic  of  a  less  than  expert  interpretation  was 
deficiency  in  ASL  or  target  language  skills.  This  being  the  case,  and 
because  the  purpose  of  the  instrument  was  to  provide  diagnostic 
feedback  to  interpreters  with  varying  degrees  of  skill  and 
experience,  it  was  important  the  instrument  encompass  ASL 
language  skills  in  addition  to  the  interpretation  skills  required  to 
interpret  from  English  to  ASL.  If  only  interpreting  was  assessed, 
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the  instrument  would  likely  have  been  less  useful  for  novice  and/or 
less  experienced  interpreters  because  many  of  their  interpretation 
errors  were  more  attributable  to  ASL  deficiencies  rather  than  actual 
interpreting  errors.  Therefore,  ASL  language  competency  was 
incorporated  into  the  assessment  domain,  but  only  within  the 
context  of  interpretation.  ASL  competency  was  never  assessed  in 
isolation,  that  is,  separate  from  the  interpreting  process. 

In  addition  to  including  ASL  language  competency  in  the 
assessment  domain  along  with  interpreting  competency,  the  task 
analysis  also  revealed  that  Deaf  consumers  required  different 
interpretations  to  match  their  variations  in  ASL  use  related  to  their 
range  in  age  and  background.  For  example,  is  the  interpreter  working 
with  Deaf  children  or  Deaf  adults?  If  they  are  adults,  are  they  high 
school  graduates  or  do  they  have  post-secondary  degrees? 
Specifically,  some  consumers  preferred  English  mouthing  while 
others  did  not.  Some  preferred  slower  signing  and  fingerspelling 
when  in  a  technical  lecture;  others  did  not.  Therefore,  it  was 
necessary  to  assess  the  degree  to  which  the  interpretation  matched 
the  Deaf  consumers'  needs  and  preferences.  This  was  also  included 
in  the  assessment  domain. 

The  task  analysis  resulted  in  a  compilation  of  379  skills  which 
detailed  the  assessment  domain  of  English-to-ASL  interpretation. 

Step  3:  Create  the  Test  Specifications  and  the  Method  of  Recording 

Observed  Behaviors  and  Step  4:  Construct  the  Initial  Pool  of  Items 

Categorizing  the  Skills 

With  the  skills  identified,  the  task  then  became  how  to  group 
the  skills  into  a  useful  format.  To  accomplish  this,  skills  were 
clustered  into  related  groupings.  For  example,  skills  such  as 
characterization,  pronominalization,  and  locatives  were  grouped 
together  under  the  heading  'Structuring  Space1  because  they  were  all 
elements  related  to  space.  Skills  such  as  non-manual  behaviors, 
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subject-object  agreement,  and  temporal  aspect  were  grouped 
together  under  the  heading  'Grammar'  because  they  all  were  related 
to  the  grammatical  principles  of  ASL.  Some  skills  were  included  in 
more  than  one  group,  for  example,  non-manual  behaviors  were  listed 
in  Classifiers/SASSes  (Size  and  Shape  Specifiers)  and  Grammar. 

Altogether,  eight  groupings  or  Major  Features  of  interpretation 
were  identified.  They  were: 

1 )  Numbers; 

2)  Fingerspelling; 

3)  Classifiers/SASSes; 

4)  Vocabulary; 

5)  Structuring  Space; 

6)  Grammar; 

7)  Interpreting;  and 

8)  Composure  and  Appearance. 

In  addition  to  assessing  individual  skills  within  each  Major 
Feature,  there  was  also  a  need  to  assess  the  interpretation  globally. 
Thus,  a  ninth  section  was  added.  This  section,  'Judgements',  allowed 
for  overall  impressions  and  comments  on  the  interpretation.  It 
allowed  for  a  narrative  summation  of  the  skills  noted  in  the 
interpretation.  For  example,  comments  like  'the  conduct  the 
interpreter  exhibits  is  appropriate'  and  'the  interpretation  would  be 
understood  by  a  deaf  ASL  consumer  (not  bilingual)'  were  placed  in 
this  section. 

Making  the  Instrument  Manageable 

As  mentioned  above,  as  a  result  of  the  task  analysis,  379 
skills  were  identified.  For  example,  a  partial  list  of  the  skills 
associated  with  interpretation  of  numbers  is  presented  in  Table  1. 
Inclusion  of  all  skills  for  each  Major  Feature  would  lead  to  an 
instrument  that  would  be  impractical  due  to  the  excessive  amount  of 
time  that  would  be  required  to  rate  an  interpretation. 


Table  1 

Partial  List  of  Skills  Related  to  Interpreting  Numbers  Accurately 
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1.  The  number  is  signed  accurately 

2.  Accurate  number  is  signed 

3.  Takes  only  one  time  to  do  it  right  (e.g.,  no  false  starts) 

4.  Numbers  expressed  in  English  are  expressed  in  ASL  (e.g., 
ability  to  alter  English  phrases  "eleven  hundred  and  two" 
into  ASL  "ONE  THOUSAND,  ONE  HUNDRED,  TWO") 

5.  Number/concept  is  shown  accurately  in  terms  of  plurality 

6.  Accurate  use  of  non  specific  number  signs  (e.g.,  few,  many) 

7.  Accurate  numbering  system 

8.  Accurate  in  signing  cardinal  numbers  (e.g.,  ONE,  TWO) 

9.  Accurate  in  signing  ordinal  numbers  (e.g.,  FIRST,  SECOND) 

10.  Accurate  in  signing  age  (e.g.,  FIVE-YEARS-OLD) 

11.  Accurate  in  signing  time/o'clock  (e.g.,  FIVE-O'CLOCK,  ONE- 
MINUTE) 

12.  Accurate  in  signing  height  (e.g.,  SIX-FEET-TALL) 

13.  Accurate  in  signs  related  to  sports  (e.g.,  MAPLE  LEAFS 
FOUR  +  OILERS  FIVE) 

14.  Accurate  in  signing  money/currency  (e.g.,  FIVE-CENTS) 

15.  Accurate  in  signing  calendar  (e.g.,  ONE-WEEK,  EVERY-YEAR) 


To  make  the  lists  more  manageable  without  deleting  skills,  the 
skills  were  aggregated  into  a  fewer  number  of  skills  which 
encompassed  the  skills  combined  without  a  loss  of  information. 
Returning  to  the  previous  example,  'interpretation  of  numbers',  the 
initial  set  of  skills  was  aggregated  into  three  skills  as  follows: 
skills  numbered  2-5  were  aggregated  with  skill  1;  skill  6  remained 
unchanged,  and  skills  numbered  8-15  were  aggregated  with  skill  7. 
Thus,  out  of  the  15  initially  identified  skills,  3  skills  were  formed: 
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the  number  is  signed  accurately;  accurate  use  of  non-specific 
number  signs  (e.g.,  FEW,  MANY);  and  accurate  numbering  system. 

This  process  was  used  for  all  of  the  identified  skills.  Table  2 
illustrates  the  number  of  original  skills  identified  in  the  task 
analysis  as  well  as  the  number  of  skills  remaining  after  aggregating 
them  within  each  Major  Feature. 


Table  2 

Number  of  Original  Skills  Aggregated  to  Lesser  Numbers  bv  Major 

Features 


Number  of  Skills 

Major  Feature _ Original  Aggregated 


Numbers 

41 

1  1 

Fingerspelling 

35 

1  0 

Classifiers/SASSes 

41 

1  3 

Vocabulary 

26 

5 

Structuring  Space 

59 

1  3 

Grammar 

63 

1  3 

Interpreting 

84 

22 

Composure  and  Appearance 

22 

5 

Judgements 

8 

7 

TOTALS 

379 

99 

Sequencing  the  Features  and  Skills 

The  eight  Major  Features  and  the  'Judgements'  section  were 
then  sequenced  to  facilitate  use.  Applying  the  research  on 
knowledge-lean  tasks  and  knowledge-rich  tasks  (e.g.,  Chi  et  al . , 
1988)  the  features  were  sequenced  beginning  with  specific  features 
(i.e.,  knowledge-lean  tasks)  required  during  portions  of  the 
interpretation  followed  by  comprehensive  features  (i.e.,  knowledge- 
rich  tasks)  that  were  required  throughout  the  interpretation.  The 
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Major  Feature  of  Numbers  was  placed  first  because  in  many 
interpretations  numbers  are  often  only  present  in  specific  segments 
of  an  interpretation.  Fingerspelling  was  placed  second. 

Fingerspelling  tends  to  be  more  common  in  most  interpretation 
samples  than  numbers.  In  contrast,  the  skills  contained  in  Grammar 
and  Interpreting  are  necessary  throughout  the  interpretation  and 
consideration  of  the  entire  interpretation  sample  would  be  required 
to  rate  them.  Therefore,  these  two  Major  Features  were  placed  sixth 
and  seventh,  respectively.  Composure  and  Appearance  was  placed 
eighth  because  these  skills  are  also  necessary  throughout  the  entire 
interpretation. 

Within  each  Major  Feature,  the  skills  were  sequenced  from 
essential  to  desirable  skills.  The  criterion  used  was  the  degree  to 
which  inaccuracy  or  inappropriateness  of  the  interpretation  was 
likely  to  affect  consumers'  comprehension  of  the  message.  When 
misunderstanding  was  the  likely  outcome  of  the  inaccurate  or 
inappropriate  interpretation,  then  the  skill  was  identified  as 
essential.  When  misunderstanding  was  not  the  likely  outcome  of  the 
inaccurate  or  inappropriate  interpretation,  the  skill  was  identified 
as  desirable.  Table  3  lists  the  number  of  essential  and  desirable 
skills  for  each  Major  Feature. 

Continuing  with  a  numbering  example,  an  essential  skill  was 
'the  number  is  signed  accurately'.  The  message  would  likely  be 
misunderstood  if  the  ASL  interpretation  was  '25'  instead  of  '250'. 
Therefore,  this  skill  was  identified  as  essential.  In  contrast,  'the 
accurate  numbering  system  is  used'  was  identified  as  desirable 
because  the  message  could  still  be  understood  accurately  even  if  the 
numbering  system  was  executed  inaccurately.  For  example,  if  the 
palm  orientation  and  the  movement  required  for  a  numbering  system 
such  as  a  person’s  height  was  not  done  accurately,  but  the  numbers 
themselves  were  signed  accurately,  then  the  message  could  still  be 
understood. 


Table  3 

The  Number  of  Essential  and  Desirable  Skills  within  Each  Major 

Feature 
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Major  Feature 

No.  of  Essential 

Skills 

No.  of  Desirable 

Skills 

Numbers 

3 

8 

Fingerspelling 

3 

7 

Classifiers/SASSes 

7 

6 

Vocabulary 

2 

3 

Structuring  Space 

7 

6 

Grammar 

6 

7 

Interpreting 

7 

1  5 

Composure  and 
Appearance 

1 

4 

Judgements 

7 

0 

Defining  the  Domain 

There  is  a  range  of  performance  from  accurate  to  inaccurate 
and  from  appropriate  to  inappropriate.  To  resolve  this  issue, 
objective  criteria  had  to  be  developed  to  determine  when  a  skill  was 
accurate  or  appropriate.  An  error  analysis  was  conducted  to 
accomplish  this  task. 

The  error  analysis  actually  took  place  at  the  same  time  as  the 
task  analysis.  In  addition  to  looking  at  the  skills  during  the  task 
analysis,  errors  were  analyzed  and  recorded.  The  result  was  the 
identification  of  a  list  of  representative  and  relevant  possible 
errors  for  each  particular  skill.  Through  this  process,  the  particular 
skill,  that  is,  the  focal  construct,  was  defined  by  the  corresponding 
list  of  possible  errors.  An  illustrative  example  is  provided  in  Table 
4  which  describes  the  momentum  of  signing  numbers.  Ideally,  when 
signing  numbers  the  momentum  should  be  smooth.  However,  if  it  is 
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not  smooth,  six  possible  types  of  errors  were  identified  that  may 
skew  the  meaning. 

Table  4 

Errors  Related  to  the  Momentum  of  Signing  Numbers 


SKILL:  Momentum  is  smooth _ 

A.  There  are  no  pauses  between  series  of  numbers  (e.g.,  5,438  & 

1,959) _ 

B.  No  pauses _ 

C.  Pauses  in  the  middle  of  a  number  (e.g.,1 ,9-pause-49  appearing 

to  be  two  separate  numbers) _ 

D.  Jerky/choppy _ 

E.  There  are  extraneous  movements  present _ 

F.  The  movement  is  incomplete _ 


To  further  describe  the  skills  and  their  associated  errors, 
examples  were  added  whenever  possible.  Keeping  in  mind  the 
instrument  was  for  use  in  North  America,  both  Canadian  and 
American  examples  were  used.  To  illustrate,  the  error  'inaccurate  in 
signs  related  to  sports'  was  accompanied  with,  'e.g.,  Maple  Leafs 
four  +  Oilers  five',  two  Canadian  teams  belonging  to  the  National 
Hockey  League. 


Summary  of  Steps  One  to  Four 

The  purpose  of  the  instrument  was  to  develop  a  practical  and 
useful  diagnostic  instrument  for  assessing  interpretations  from 
English  to  ASL. 

The  domain  was  defined  and  characterized  by  including  both 
ASL  and  interpreting  skills  because  these  two  competencies  were 
interrelated. 
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The  outcome  of  the  instrument  development  phase  of  the 
research  was  a  diagnostic  instrument  consisting  of  eight  Major 
Features.  Each  feature  contained  a  list  of  skills,  or  focal 
constructs,  with  each  skill  followed  by  one  or  more  representative 
and  relevant  possible  errors.  The  collection  of  errors  for  a 
particular  skill  provided  an  definition  of  that  skill. 

In  addition  to  the  eight  Major  Features,  a  ninth  category, 
'Judgements',  was  added  to  allow  a  global  assessment  of  the 
interpretation  sample.  The  Major  Features,  'Judgements',  and  the 
skills  within  each  feature  were  sequenced  for  ease  of  use  by  the 
rater. 

The  result  of  completing  this  phase  of  the  research  was  the 
initial  draft  of  the  diagnostic  assessment  instrument  for  English- 
to-ASL  interpretations. 


Panel  Review 

Step  5:  Review  of  the  Instrument  By  a  Panel  of  Interpretation 

Experts 

Following  the  initial  development  of  the  instrument,  a  panel  of 
14  experts  reviewed  it.  The  purpose  of  this  review  was  to  validate 
the  Major  Features  and  the  skills  with  their  associated  errors 
within  each  Major  Feature.  The  experts  and  the  procedures  they 
followed  are  described  in  the  balance  of  this  chapter. 

Selecting  the  Panel 

Panel  members  were  chosen  on  the  basis  of  their  expertise  in 
the  field  of  interpreting  and  either  their  participation  at  the  annual 
meeting  of  the  Canadian  Association  for  the  Education  of  Sign 
Language  Interpreters  (CAESLI)  or  their  proximity  to  the  meeting 
location.  CAESLI  is  an  organization  of  interpreter  educators  which 
attracts  participants  from  across  Canada. 
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CAESLI  was  established  in  1985  to  provide  a  forum  for 
Canadian  interpreter  educators  to  meet  and  discuss  common  issues 
within  their  programs  with  the  ultimate  goal  of  improving 
interpreting  programs  across  the  country.  New  programs  were  being 
established  and  educators  recognized  that  there  was  a  need  to  meet 
regularly  and  share  materials  and  curriculum  to  make  the  limited 
available  resources  extend  to  their  fullest. 

Hearing  and  Deaf  experts  who  would  be  attending  the  annual 
CAESLI  meeting  in  Winnipeg,  Manitoba,  October  9  to  11,  1991  were 
contacted  via  electronic  mail  and  invited  to  participate  on  an  expert 
review  panel  for  a  newly  developed  instrument  designed  to  assess 
English-to-ASL  interpretations.  They  were  given  the  day,  time  and 
location  of  the  meeting  which  coincided  with  the  CAESLI  meeting 
but  did  not  conflict  with  any  CAESLI  function.  Invitations  were  also 
issued  to  interpreter  educators  who  lived  in  Winnipeg  but  were  not 
attending  the  CAESLI  meetings. 

Altogether  15  invitations  were  issued  and  all  15  people  agreed 
to  participate  in  the  research.  Subsequently,  one  was  unable  to 
complete  the  task  due  to  personal  conflicts.  Thus,  the  results  cited 
in  this  step  of  the  validation  process  are  based  on  the  work  of  14 
expert  panelists. 

All  the  experts  resided  in  Canada.  None  of  them  had  prior 
involvement  with  developing  the  instrument.  Collectively  they  were 
interpreter  educators  and/or  administrators  representing  six 
different  interpreting  programs  from  across  Canada:  Douglas 
College  in  Vancouver,  British  Columbia,  Grant  MacEwan  Community 
College  in  Edmonton,  Alberta,  Red  River  Community  College  in 
Winnipeg,  Manitoba,  Sheridan  College  of  Applied  Arts  and  Technology 
in  Brampton,  Ontario,  St.  Clair  College  of  Applied  Arts  and 
Technology  in  Windsor,  Ontario  and  Universite  du  Quebec  a  Montreal, 
Quebec. 
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Hearing  Experts'  Qualifications 

The  experts  completed  a  form  (see  Appendix  C)  outlining  their 
expertise.  As  shown  in  Table  5,  eight  of  the  fourteen  experts  who 
participated  in  the  panel  review  validation  stage  of  the  instrument 
were  hearing  and  all  eight  were  actively  interpreting.  They  ranged 
in  age  from  30  to  45  years,  with  a  mean  age  of  36.6  years.  Six  of 
the  eight  were  ASL/English  interpreters.  One  was  LSQ/French  and 
one  was  a  Spanish/English  interpreter.  English  was  the  native 
language  of  six  experts.  One  of  the  experts  was  a  native  French 
speaker  and  one  was  a  native  ASL  user. 

All  except  one  had  experience  teaching  interpreters.  Their 
experience  ranged  from  3  to  15  years,  with  a  mean  of  7.3  years. 
Seven  of  the  experts  had  experience  teaching  in  interpreting 
programs  with  six  of  them  also  having  experience  in  teaching 
continuing  education  courses  and/or  workshops  to  professional 
interpreters. 

Three  people  held  national  certification,  the  Comprehensive 
Skills  Certificate  (CSC),  from  the  Registry  of  Interpreters  for  the 
Deaf  in  the  United  States.  One  of  these  people  also  held  the 
Certificate  of  Interpretation  (COI)  from  the  Association  of  Visual 
Language  Interpreters  of  Canada.  A  fourth  expert  held  only  the  COI. 
Of  the  remaining  four  people,  two  interpreted  for  languages  where 
no  equivalent  certification  exam  existed,  that  is,  Spanish/English 
and  LSQ/French,  and  two,  who  were  involved  in  developing  the  COI 
for  AVLIC,  were  not  able  to  undergo  the  certification  process 
because  of  their  involvement. 

In  terms  of  membership  in  local  and  national  organizations, 
seven  belonged  to  at  least  one  local  interpreting  organization  (i.e. , 
Ontario  Association  of  Sign  Language  Interpreters,  Western 
Association  of  Visual  Language  Interpreters,  Manitoba  Association 
of  Visual  Language  Interpreters,  Alberta  Chapter  of  the  Registry  of 
Interpreters  for  the  Deaf,  and  Association  Quebeqois  des  Interpretes 
Francophones  en  Langage  Visuel.)  All  belonged  to  AVLIC,  with  two 


Table  5 

Hearing  Experts1  Qualifications 


(N=8) 


Interpreting  experience 

Actively  interpreting 

8 

ASL/English  interpreters 

6 

LSQ/French 

1 

Spanish/English 

1 

Native  language 

English 

6 

French 

1 

ASL 

1 

Experience  teaching  interpreters 

Experience  teaching 
interpreting  students  and 
professional  interpreters 

6 

Experience  teaching  only 
interpreting  students 

1 

No  experience  teaching 
interpreters 

1 

Certification(s)  held 

CSC  and  COI 

1 

COI 

1 

CSC 

2 

Not  applicable 

4 

Membership  in  organizations 

Local  interpreting 
organizations 

7 

CAESLI  and  CIT 

4 

CAESLI  only 

2 

AVLIC  and  RID 

2 

AVLIC  only 

6 
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also  belonging  to  RID.  Six  of  the  eight  belonged  to  CAESLI  with  four 
belonging  to  both  CAESLI  and  its  sister  organization,  the  Conference 
of  Interpreter  Trainers  (CIT)  in  the  United  States. 

Deaf  Experts1  Qualifications 

Six  of  the  fourteen  member  panel  were  Deaf.  All  six  were 
native  ASL  users  with  one  native  bilingual  in  ASL  and  English.  Their 
ages  ranged  from  30  to  45  years,  with  a  mean  of  36.6  years.  They 
possessed  from  13  to  20  years  of  experience  using  interpreters, 
with  a  mean  of  17.3  years.  They  all  had  experience  using  seasoned 
interpreters,  and  all  except  one  had  experience  using  new 
interpreters. 

All  had  experience  teaching  ASL  to  hearing  adults  who  were 
not  interpreters.  Five  also  had  experience  teaching  ASL  to 
interpreters  and  three  had  experience  teaching  ASL  to  children.  The 
range  in  teaching  experience  was  from  1.5  to  16  years,  with  a  mean 
of  7.9  years. 

All  six  belonged  to  at  least  one  local  deaf  organization  (e.g. , 
Edmonton  Association  of  the  Deaf,  Alberta  Cultural  Society  of  the 
Deaf,  British  Columbia  Cultural  Society  of  the  Deaf,  Greater 
Vancouver  Association  of  the  Deaf,  or  Western  Canadian  Association 
of  the  Deaf).  Two  belonged  to  two  local  deaf  organizations  with  one 
of  them  belonging  to  three.  All  except  one  belonged  to  at  least  one 
national  organization,  either  the  Canadian  Association  of  the  Deaf 
(CAD)  or  the  Canadian  Cultural  Society  of  the  Deaf  (CCSD)  with  two 
of  them  belonging  to  both.  A  summary  of  the  qualifications  of  the 
Deaf  members  of  the  panel  is  provided  in  Table  6. 


Table  6 

Deaf  Experts'  Qualifications 
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(N=6). 


Qualifications 

Native  language(s) 

ASL  only 

5 

Bilingual  ASL  and  English 

1 

Experience  using  interpreters 

Seasoned  interpreters  and 

5 

new  interpreters 

Only  seasoned  interpreters 

1 

Experience  teaching  ASL  to 

Hearing  adults,  interpreters 

3 

and  children 

Hearing  adults  and 

2 

interpreters 

Only  hearing  adults 

1 

Membership  in  deaf  organizations 

3  local  organizations 

1 

2  local  organizations 

2 

1  local  organization 

3 

CAD  and  CCSD 

2 

CAD  and/or  CCSD 

3 

Meeting  With  the  Experts 

Prior  to  the  experts  actually  reviewing  the  instrument,  the 
researcher  met  with  the  panel  members  to  describe  their  task  in  the 
first  validation  process,  the  instrument  development  procedures  to 
date,  and  the  future  validation  procedures  planned.  Briefly,  the 
panel  members  were  asked  to  conduct  a  line-by-line  review  of  the 
instrument  and  told  that  this  was  the  first  step  in  the  overall 
validation  process.  The  purpose  of  the  instrument  was  then 


described.  The  description  outlined  the  need  to  assess  English-to- 
ASL  interpretations  and  to  provide  diagnostic  feedback  to 
interpreters  with  varying  degrees  of  skills  and  experience.  The 
basic  premise  was  that  at  any  time  an  interpreter  could  be  an  expert 
or  a  novice  depending  on  the  setting  and  a  multitude  of  other  factors 
such  as  familiarity  with  the  content  of  the  message  and  the  pace  of 
the  speaker. 

The  general  format  of  the  instrument  was  then  described.  It 
was  explained  that  the  instrument  assessed  both  ASL  and 
interpreting  skills,  and  that  the  ASL  skills  were  only  assessed 
within  the  context  of  interpretation. 

In  addition,  the  components  of  the  instrument  which  were  not 
completed,  but  which  would  be  included  at  a  future  date,  were 
described.  These  components  included  the  addition  of  definitions 
corresponding  to  each  skill  and  a  rating  scale  which  would  be  used 
to  assess  the  level  of  performance.  Lastly,  the  next  steps  in  the 
validation  process  were  outlined  for  the  panel  members. 

It  was  not  possible  for  all  of  the  invited  experts  to  meet  at 
the  same  time.  The  researcher  met  with  eleven  of  the  fourteen 
experts  in  a  three-hour  open  forum  format.  The  remaining  three 
were  consulted  individually,  at  their  convenience. 

The  open  forum  with  eleven  experts  was  conducted  in  ASL 
because  this  was  the  language  most  common  among  them.  One  of  the 
experts  interpreted  into  English  for  two  whose  languages  were  other 
then  ASL.  Following  the  introductory  remarks  and  explanation  of  the 
work  they  were  to  perform,  the  researcher  answered  their  questions. 

Just  prior  to  beginning  their  review,  they  were  informed  that 
their  names  would  be  held  in  confidence  and  that  no  individual 
results  would  be  reported.  All  of  their  comments  and  suggestions 
would  be  grouped  into  summary  form,  thus  allowing  the  experts 
more  freedom  to  provide  honest  feedback  without  worrying  about 
their  comments  being  associated  with  their  names.  They  were 
encouraged  to  be  critical  in  their  review  of  the  instrument. 
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Specification  of  the  Task  for  the  Panel  Members 

The  responsibilities  of  the  panel  members  consisted  of 
validating  the  instrument  by  using  the  rating  forms  provided  (see 
Appendix  D).  The  components  to  be  evaluated  included  the 
-list  of  Major  Features; 

-lists  of  skills  within  each  Major  Feature; 

-lists  of  possible  errors  associated  with  each  skill; 
-sequencing  of  the  skills;  and 
-the  clarity  of  the  writing. 

The  first  task  of  the  experts  was  to  examine  the  list  of  Major 
Features  using  the  Major  Features  Rating  Form.  They  were  asked  to 
determine  whether  the  Major  Features,  as  a  set,  sufficiently 
described  the  assessment  domain  of  interpretation  from  English  to 
ASL.  Panel  members  were  asked  to  add  Major  Features  they  thought 
were  pertinent  to  the  assessment  domain  or  to  delete  Major 
Features  that  were  not  essential  to  the  assessment  domain. 

The  panel  then  examined  the  content  of  each  Major  Feature 
using  the  Expert  Rating  Forms.  One  form  was  used  for  each  Major 
Feature.  They  were  to  determine  whether  each  identified  skill  and 
accompanying  errors  within  each  Major  Feature  were  relevant  to  the 
Major  Feature.  They  were  then  asked  whether  the  set  of  skills  and 
their  errors  was  representative  of  the  Feature. 

The  experts  were  also  asked  to  differentiate  between 
essential  skills  and  desirable  skills  within  each  feature.  The 
criterion  for  differentiating  these  was  the  degree  to  which 
inaccuracy  or  inappropriateness  of  the  interpretation  was  likely  to 
affect  the  consumers'  comprehension  of  the  message.  Essential 
skills  were  defined  as  those  skills  that  had  to  be  executed 
accurately  and/or  appropriately  in  order  to  have  equivalency  of 
meaning  between  the  English  source  language  and  the  ASL 
interpretation  to  comprehend  the  meaning.  Desirable  skills  were 
defined  as  those  skills  which  if  executed  inaccurately  or 


4  9 


inappropriately,  might  skew  the  message  but  not  likely  result  in 
misunderstandings. 

They  were  also  asked  to  examine  the  sequencing  of  the  skills 
listed  within  each  Major  Feature.  The  skills  were  listed  from  most 
essential  to  least  desirable.  If  they  disagreed  with  the  sequence 
presented,  then  they  were  asked  to  reorder  the  skills  to  what  they 
thought  was  a  more  appropriate  sequence. 

Finally,  they  were  asked  to  review  the  writing  of  each  item  for 
its  technical  adequacy  and  freedom  from  bias,  taking  particular  note 
of  whether  the  terminology  was  accurate,  current,  and  appropriate. 

Panel  Review  Procedures 


The  panel  members  who  attended  the  meeting  worked 
individually  in  the  same  room.  They  were  encouraged  to  ask 
questions  or  to  seek  clarification.  They  were  asked  not  to  confer 
with  each  other  or  to  ask  questions  of  anyone  except  the  researcher. 

Sign  Language  books,  interpreting  books,  and  reference 
materials  used  within  the  interpreting  profession  both  by 
interpreting  educators  and  professional  interpreters  were  available 
for  them  to  use.  Most  of  the  resources,  if  not  all,  were  very  familiar 
to  the  panel  members. 

Three  panel  members  needed  additional  time  to  complete  their 
work;  all  returned  their  completed  instrument  at  a  later  date  either 
in  person  or  by  mail.  The  remaining  eight  completed  their  work  by 
the  end  of  the  meeting. 

The  three  panel  members  who  were  unable  to  attend  the  open 
forum  worked  individually  after  receiving  instructions  in  person 
from  the  researcher. 

Debriefing  was  conducted  either  at  the  end  of  the  meeting  for 
those  who  were  in  attendance  or  at  a  personal  meeting  when  the 
work  was  completed  individually.  The  researcher  reminded  the  panel 
members  to  keep  confidential  the  information  related  to  the 
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instrument  because  at  a  future  date  other  experts  in  the  field  would 
further  validate  of  the  instrument. 

Analysis  of  Validation  Panel  Data 

The  data  from  the  fourteen  members  of  the  validation  panel 
were  aggregated  for  each  Major  Feature,  skill,  and  error.  Written 
comments  were  color  coded  to  distinguish  Deaf  from  hearing 
experts.  This  allowed  cross-referencing  on  each  item  and  provided  a 
means  to  assess  the  consistency  among  panel  members  within  and 
between  the  hearing  and  Deaf  groups. 

Comments  were  also  numbered  to  identify  them  by  panel 
member  so  that  the  researcher  could  request  further  clarification,  if 
needed.  As  it  turned  out,  the  comments  from  all  of  the  experts  were 
clearly  articulated  and  there  was  no  need  to  ask  for  further 
clarification. 


Results  of  the  Panel  Review 
Panelists'  General  Comments 

Panel  members  indicated  that  the  instrument,  as  planned, 
would  fill  a  necessary  gap  in  assessment  methods  currently 
available  for  English-to-ASL  interpretation.  As  well,  they  stated 
that  the  instrument  was  comprehensive,  and  that  it  had  a  wide  range 
of  possible  uses. 

Major  Features  Revisions 

Twelve  of  the  fourteen  experts  agreed  that  all  of  the  Major 
Features  were  necessary.  The  two  who  did  not  agree  thought  that 
Grammar  should  be  incorporated  into  Interpreting  and  that 
Structuring  Space  should  be  divided  into  Spatial  Relationships  and 
ASL  Structure.  Grammar  was  specifically  taken  out  of  interpreting 
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so  as  to  rate  the  command  of  language,  as  separately  as  possible, 
from  the  actual  interpreting  task.  Spatial  relationships  were 
already  a  sub-component  of  the  Major  Feature  Structuring  Space  and 
ASL  structure  was  already  included  within  Grammar.  Thus,  the 
Major  Features  were  kept  as  originally  written  with  one  exception; 
the  title  of  the  last  section,  Judgement,  was  changed  to  Rater's 
Impressions.  Several  of  the  panelists  suggested  that  the  title  was 
more  appropriate  given  the  nature  of  the  rating  task. 

Sequence  of  Major  Features  Revisions 

Based  on  the  data  provided  by  the  panel  review,  the  sequencing 
of  the  Major  Features  was  altered  as  noted  in  Table  7.  As  shown, 
Numbers  and  Fingerspelling,  Classifiers/SASSes  and  Vocabulary,  and 
Structuring  Space  and  Grammar  were  reversed.  No  feature  was 
displaced  further  than  one  Major  Feature  away  from  its  original 
position.  The  last  two  Major  Features,  Interpreting  and  Composure 
and  Appearance,  remained  in  their  original  order,  followed  by  the 
last  section,  Rater's  Impressions. 

Table  7 

Major  Features  Sequence  Revisions  as  a  Result  of  the  Panel  Review 


Initial  Sequence 

Revised  Sequence 

Numbers 

Fingerspelling 

Fingerspelling 

Numbers 

Classifiers/SASSes 

Vocabulary 

Vocabulary 

Classifiers/SASSes 

Structuring  Space 

Grammar 

Grammar 

Structuring  Space 

Interpreting 

Interpreting 

Composure  and  Appearance 

Composure  and  Appearance 

Judgements 

Rater's  Impressions 
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Skills  and  Errors  Revisions 

As  a  result  of  the  panel  review,  the  lists  of  skills  and  errors 
within  each  Major  Feature  were  considerably  revised.  A  summary  of 
these  revisions  is  provided  in  Table  8. 

Table  8 

Revisions  of  Skills  and  Errors  within  Major  Features  as  a  Result  of 

the  Panel  Review 


Major 

Feature 

Additions 

Deletions 

Sequence 

Changes 

Editorial 

Aggre¬ 

gated 

T  otals 

Numbers 

7 

6 

24 

1  2 

1  7 

66 

Finger- 

spellinq 

1  4 

3 

1  3 

1  1 

1  3 

54 

Classifiers 

&  SASSes 

4 

2 

1  5 

20 

1  3 

54 

Vocabulary 

1  7 

0 

4 

4 

1  1 

36 

Structur¬ 
ing  Space 

4 

3 

32 

1  6 

1  9 

74 

Grammar 

1  4 

8 

1  2 

6 

42 

82 

Interpret¬ 

ing 

21 

7 

41 

32 

24 

125 

Composure 

Appearance 

6 

0 

1  4 

6 

3 

29 

TOTALS 

87 

29 

1  55 

107 

142 

520 

The  Major  Feature  of  Interpreting  had  the  greatest  number  of 
changes  with  125  revisions.  Grammar  had  the  second  highest 
number  of  changes,  with  a  total  of  82  revisions.  The  least  number  of 
changes  were  to  Vocabulary  (36)  and  Composure  and  Appearance 
(29). 
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Across  all  eight  of  the  Major  Features,  the  greatest  number  of 
revisions  was  related  to  the  sequencing  of  the  skills  and  errors.  Of 
the  155  sequence  changes,  129  items  moved  within  the  same  Major 
Feature,  with  the  remaining  26  moved  to  different  Major  Features. 

A  number  of  skills  and  errors  were  aggregated  with  other 
skills  and  errors.  Of  the  142,  111  were  aggregated  within  the  same 
Major  Feature,  with  the  remaining  31  aggregated  with  skills  and 
errors  within  other  Major  Features. 

In  the  last  section  of  the  instrument,  Judgements,  there  was  a 
total  of  11  revisions:  4  additions,  1  deletion,  3  editorial  changes, 
and  3  aggregated  with  other  items  within  Judgements. 

Additions  to  Complete  the  Instrument 


Definitions 


Definitions  were  added  corresponding  to  each  skill  with  its 
related  list  of  possible  errors  to  clarify  their  meaning  and  to  assist 
raters  in  the  assessment  process.  The  definitions  were  numbered 
with  the  same  number  of  the  skill  they  defined.  They  were  printed 
on  the  left  page  corresponding  to  the  appropriate  skill  on  the  right 
page.  Depending  on  the  length  of  the  definitions,  and  the  length  of 
the  skills  and  errors  they  defined,  one  to  three  skills  were  on  each 
of  the  right  pages.  The  number  of  skills  and  definitions  never 
exceeded  three  per  page.  This  meant  the  user  would  read  the  first 
definition  on  the  left  page  and  then,  before  moving  down  the  page  to 
the  second  definition,  would  read  the  first  skill  with  its  list  of 
possible  errors  on  the  right  page.  Then,  for  the  second  skill,  go  back 
to  the  left  page  continuing  in  the  same  manner. 

With  the  definitions  in  place  and  with  the  goal  of  providing 
interpreters  with  diagnostic  feedback,  blank  lines  were  added  after 
each  list  of  possible  errors  to  allow  space  for  the  rater  to  make 
specific  comments  on  the  interpretation  performance  related  to 
each  skill. 
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Rating  Scale 

The  purpose  of  the  rating  scale  was  for  the  rater  to  identify 
the  degree  to  which  the  skill  was  executed  accurately  or 
appropriately. 

Two  rating  scales  were  used.  The  first,  to  be  used  at  the  skill 
level,  consisted  of  three  options:  1)  Y-yes,  the  skill  was  performed 
accurately  or  appropriately,  2)  N-No,  the  skill  was  not  performed 
accurately  or  appropriately,  and  3)  NT-Not  in  Target  or  not  present 
in  the  interpretation  and  therefore  unable  to  rate  (i.e.,  not 
applicable).  The  second  scale  for  rating  the  error  level,  consisted  of 
4  ordered  points:  from  1  -the  skill  was  performed  well  to  4-meaning 
the  skill  was  performed  poorly,  with  points  2  and  3  in  the  middle  of 
the  two  extremes. 


Instrument 


A  Table  of  Contents  was  added  with  the  eight  Major  Features 
and  the  Rater's  Impressions.  This  became  the  second  page  of  the 
instrument  preceded  by  the  title  page. 

Following  the  panel  review  validation  process  and  the 
revisions  made  to  the  instrument,  it  was  used  in  two  field  tests. 

The  procedures  of  and  the  results  from  these  field  tests  are 
described  in  the  following  two  chapters;  Chapter  4:  Field  Test  1  and 
Chapter  5:  Field  Test  2. 
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CHAPTER  4:  FIELD  TEST  1 

Following  the  development  of  the  instrument,  two  field  tests 
were  conducted.  The  first,  Field  Test  1,  included  two  raters  from 
Alberta,  Canada.  The  second,  Field  Test  2,  included  seven  raters 
from  California,  United  States. 

Both  Field  Test  1  and  Field  Test  2  were  preliminary  tryouts. 
The  raters'  responsibilities  were  to  validate  the  definitions,  further 
validate  the  skills  and  errors  within  each  Major  Feature,  and  for  the 
first  time  actually  assess  interpretation  samples  using  the 
instrument.  The  raters  in  Field  Test  2  also  conducted  a  line-by-line 
review  of  the  rater's  Instruction  Manual  which  was  developed  after 
Field  Test  1 . 

Described  in  this  chapter  are  the  methods  used  for  and  results 
from  Field  Test  1.  The  corresponding  information  for  Field  Test  2  is 
presented  in  the  following  chapter. 

Method 

Developing  the  Audio  Tape  Stimulus 

An  English  stimulus  audio  cassette  tape  was  developed  for  use 
in  the  field  tests.  Using  this  tape  as  the  stimulus,  two  interpreters 
interpreted  a  spoken  English  message  into  ASL  while  being  video 
taped  (described  in  the  next  section).  The  message  contained  on  the 
audio  tape  was  developed  to  solicit  an  ASL  interpretation  that 
required  a  wide  variety  of  interpretation  skills,  thus  providing 
samples  of  typically  found  skills  that  could  be  rated  using  the 
instrument.  It  was  specifically  designed  to  include  English  text 
which  required  the  use  of  skills  from  all  of  the  Major  Features 
included  in  the  instrument. 

The  speaker  was  female.  She  was  asked  to  develop  a  formal 
lecture,  approximately  five  minutes  in  length,  that  included  all  eight 
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Major  Features.  A  verbatim  transcript  of  this  lecture  is  provided  in 
Appendix  E. 


The  Interpreters 

Two  interpreters  agreed  to  interpret  the  English  audio  tape 
stimulus  while  being  video  taped  and  to  then  have  their  video  taped 
interpretations  used  for  the  field  tests.  The  interpreters  were 
chosen  to  reflect  different  skill  levels,  thereby  providing  a  test  of 
the  sensitivity  of  the  instrument  to  different  performance  levels  of 
interpretation.  Both  were  hearing  interpreters  residing  in  Alberta. 
English  was  their  native  language.  They  learned  American  Sign 
Language  as  a  second  language  through  both  deaf  and  hearing  people. 
They  attended  and  graduated  from  the  same  one  year  (35  hours  per 
week)  interpreting  program.  In  addition  to  the  one  year  interpreting 
program,  Interpreter  1  had  one  year  of  post-secondary  education  and 
Interpreter  2  completed  a  university  undergraduate  degree.  Neither 
had  obtained  certification  from  AVLIC  (Association  of  Visual 
Language  Interpreters  of  Canada)  or  RID  (Registry  of  Interpreters  for 
the  Deaf). 

Interpreter  1  had  five  and  a  half  years  of  interpreting 
experience,  while  Interpreter  2  possessed  two  years  of  experience. 
Both  freelanced  between  15  and  20  hours  a  week,  primarily  in 
employment,  medical,  and  post-secondary  education  settings.  They 
interpreted  for  deaf  people,  some  of  whom  were  native  signers  and 
others  who  had  learned  ASL  later  in  life.  Both  were  members  of 
their  local  and  national  interpreting  organizations  and  both  attended 
between  60  and  75  hours  of  workshops  annually. 

Developing  the  Video  Tape 

Prior  to  video  taping,  the  interpreters  were  provided  with 
written  directions  and  told  that  they  were  interpreting  in  a  formal 
setting  for  a  Deaf  male,  about  35  years  of  age,  whose  native 
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language  was  ASL.  They  were  also  told  that  the  audio  tape  was  five 
minutes  long  and  included  text  which  required  a  variety  of  ASL 
techniques  such  as  fingerspelling,  numbers,  structuring  space,  and 
classifiers. 

After  reading  the  instructions,  each  interpreter  first  listened 
to  the  first  minute  of  the  audio  tape.  After  the  audio  tape  was 
rewound,  they  then  interpreted  the  English  message  into  ASL.  Even 
though  they  heard  the  first  minute,  their  unrehearsed  or  "cold" 
interpretation  was  similar  to  the  first  five  minutes  of  any  formal 
lecture  with  a  minimal  amount  of  information  given  to  the 
interpreter  prior  to  actually  interpreting  it. 

The  interpreters  were  taped  on  different  days,  onto  two 
different  video  tapes.  The  audio  tape  player  was  set  at  a 
comfortable  volume  for  the  interpreter  to  hear  and  also  loud  enough 
for  the  video  tape  recorder  to  record  the  sound,  thereby  recording 
the  speaker.  Once  the  interpretation  began,  the  researcher  left  the 
room  so  as  to  avoid  undue  pressure  brought  on  by  an  external 
observer. 


Using  the  Instrument  and  Rating  Performances 

The  Raters 


Two  raters  participated  in  this  step  of  the  validation  process 
and  provided  detailed  feedback  about  the  content  and  the  utility  of 
the  instrument.  The  raters  were  selected  on  the  basis  of  their 
expertise  in  the  area  of  interpreter  education  and  their 
demonstrated  ability  to  assess  interpretations  and  use  interpreting 
assessment  procedures. 

Both  raters  were  from  Canada  and  were  living  in  Alberta.  They 
were  both  female  in  their  thirties.  Both  were  native  English 
speakers  and  one,  Rater  1,  was  also  a  native  ASL  user. 

Both  raters  were  hearing  with  five  or  more  years  of  teaching 
experience.  In  each  case,  their  experience  included  teaching 
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students  within  a  post-secondary  interpreting  program  and  teaching 
workshops  and  continuing  education  classes  to  professional 
interpreters.  Rater  1  held  both  the  Certificate  of  Interpretation 
from  AVLIC  and  the  Comprehensive  Skills  Certificate  offered  by  RID, 
while  Rater  2  held  the  Comprehensive  Skills  Certificate  from  RID. 
Both  were  members  of  the  Canadian  Association  for  the  Education  of 
Sign  Language  Interpreters  (CAESLI)  and  the  Conference  of 
Interpreter  Trainers  (CIT)  in  the  United  States.  They  were  also 
members  of  AVLIC,  RID,  the  Association  of  Visual  Language 
Interpreters  of  Southern  Alberta,  and  the  Alberta  Chapter  of  the 
Registry  of  Interpreters  for  the  Deaf.  Neither  rater  had  had 
involvement  with  any  of  the  prior  steps  in  the  development  of  the 
instrument. 


Procedures  for  Rating  Performances 

The  raters  agreed  to  hold  two  meetings  with  the  researcher 
over  a  two  day  period  in  the  Spring  of  1992.  The  first  meeting  was 
held  with  both  raters  present  and  the  second  meeting  was  held 
individually  with  each  rater. 

Meeting  With  Raters  Together 

The  researcher  met  with  the  two  raters  for  approximately 
three  hours  to  discuss  the  instrument  prior  to  their  using  it.  At  this 
meeting,  each  was  provided  with  a  package  of  materials.  The 
package  for  each  rater  included  a  copy  of  the  two  video  tape 
interpretation  samples  and  three  copies  of  the  second  draft  of  the 
instrument.  One  copy  of  the  instrument  was  to  be  used  by  the  raters 
to  record  their  written  comments  and  the  other  two  copies  were  to 
be  used  to  rate  the  two  interpretation  performances. 

The  goal  of  the  instrument  was  described  to  the  raters  as  a 
"diagnostic  assessment  of  interpretation  performances  from  spoken 
English  to  ASL."  The  instrument  was  then  described  in  detail  as 
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follows.  Eight  Major  Features  of  interpretation  were  identified 
through  a  task  analysis  and  an  expert  panel  review.  The  Major 
Features  were  followed  by  a  section  for  overall  assessment.  For 
each  Major  Feature,  related  skills  were  listed,  with  each  skill 
followed  by  a  list  of  one  or  more  possible  errors.  Both  the  Major 
Features  and  the  skills  within  each  feature  were  sequenced  into  a 
logical  order  from  knowledge-lean  tasks  to  knowledge-rich  tasks. 
Each  list  of  errors  was  followed  by  a  series  of  blank  lines  in  which 
the  rater  could  document  feedback  about  the  interpretation  which 
would  then  be  provided  to  the  interpreter.  Each  skill  with  its  list  of 
possible  errors  had  a  corresponding  definition  on  the  opposite  page. 

It  was  pointed  out  that  the  definitions  were  still  at  an  initial  stage. 

The  anticipated  future  appearance  of  the  instrument  was  then 
described.  They  were  told  it  would  be  spiral-bound  with  each  Major 
Feature  printed  on  a  different  color  of  paper  to  separate  one  feature 
from  another.  The  pages  and  the  skills  would  be  numbered 
consecutively  from  beginning  to  end.  Also,  an  instruction  manual  for 
the  rater  would  be  included  with  the  instrument. 

After  describing  the  instrument  to  the  raters,  they  reviewed  it 
with  the  researcher  present.  The  researcher  answered  their 
questions. 

The  raters  were  told  to  start  their  assessment  at  the 
beginning  of  the  instrument  after  which  they  could  proceed  in  any 
manner  that  was  comfortable  for  them.  For  example,  they  could 
assess  each  feature  completely  before  proceeding  to  the  next  Major 
Feature  or  they  could  proceed  rating  specific  skills  and  errors  as 
they  appeared  in  the  interpretation. 

The  raters  were  also  instructed  to  view  the  video  tape  at  least 
once  without  sound.  Viewing  without  sound  provides  a  different 
perspective  from  receiving  both  a  visual  and  auditory  message 
simultaneously.  This  could  be  done  at  any  time  during  the 
assessment  and  it  was  left  up  to  the  raters  to  decide  when  and  how 
many  times  they  needed  to  view  the  sample  without  sound. 
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Assessment  of  the  Sample  Interpretations 

The  raters  worked  separately  in  their  own  homes.  This 
provided  for  a  relaxed  and  comfortable  environment  in  which  to 
complete  the  assessments.  Each  rater  had  her  own  television 
monitor  and  video  playback  unit  with  a  remote  control.  The 
assessments  were  done  at  different  times,  allowing  the  researcher 
to  observe  each  rater  while  she  assessed  the  video  taped 
interpretations.  Rater  1  was  observed  while  conducting  both 
assessments.  Due  to  time  limitations,  Rater  2  was  observed  only 
during  her  first  assessment;  the  rating  took  longer  than  anticipated 
and  thus  there  was  not  sufficient  time  available  to  observe  Rater  2 
assess  both  interpretations.  Rater  2  completed  her  second 
assessment  on  her  own  and  returned  it  with  the  video  tape  to  the 
researcher. 

Both  raters  viewed  the  two  video  tapes  in  the  same  sequence, 
Interpretation  1  followed  by  Interpretation  2. 

Collecting  Raters1  Reactions  to  the  Assessment  Procedures 

Following  completion  of  the  first  rating,  the  raters  were  asked 
a  number  of  questions  regarding  the  assessment  process.  First,  each 
was  asked  to  provide  general  comments.  Then  each  was  asked  more 
specific  questions  on  the  content,  the  utility,  and  the  flow  of  the 
instrument.  They  were  asked  to  provide  feedback  on  the  stimuli  and 
the  rating  procedures.  The  researcher  also  held  discussions  with 
Rater  1  after  she  completed  her  second  assessment. 

In  addition  to  the  responses  to  these  questions,  the  researcher 
took  notes  during  the  two  assessments  completed  by  Rater  1  and  the 
first  assessment  completed  by  Rater  2.  The  time  required  to 
complete  each  assessment  and  the  way  each  rater  worked  through 
the  instrument  was  also  recorded.  Rater  2,  who  did  the  second 
assessment  on  her  own,  documented  her  start  and  end  time  for  her 
second  assessment. 
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Findings 

Time  Required  to  Complete  the  Assessments 

Rater  1  required  2  hours  and  10  minutes  to  complete  her 
assessment  of  Interpretation  1  and  1  hour  and  55  minutes  to  assess 
Interpretation  2.  She  reported  that  the  instrument  was  much  easier 
to  use  the  second  time.  The  longer  time  for  the  first  interpretation 
may  be  accounted  for  by  the  observation  that  in  addition  to 
completing  the  assessment,  she  also  made  written  comments  on  the 
instrument's  utility  and  the  clarity  of  the  writing.  Rater  2  required 
1  hour  and  30  minutes  to  complete  each  of  her  two  interpretation 
assessments. 


Raters'  General  Comments 


Both  raters  stated  that  the  instrument  provided  a  way  to 
standardize  observations  of  interpretations  and  commented  the 
instrument  was  very  detailed.  They  suggested  that,  because  of  this 
detail,  they  looked  for  and  analyzed  skills  and  errors  different  from 
those  they  typically  assessed  in  their  individual  work  with 
interpreters.  They  felt  the  instrument  provided  the  structure  and 
detail  to  examine  a  wide  variety  of  skills  and  errors. 

Both  raters  commented  particularly  on  the  items  within  the 
last  section,  Rater's  Impressions.  The  raters  indicated  that  the 
questions  in  this  section  were  very  valuable  and  allowed  them  to 
comment  on  the  work  in  a  more  global  assessment  than  the  eight 
Major  Features.  They  felt  this  section  provided  a  good  summary  of 
the  interpreters'  work. 
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Specific  Reactions  to  the  Instrument 


Major  Features 

The  raters  commented  that  the  instrument  was  easy  to  work 
through.  Both  raters  indicated  that  the  sequence  of  the  Major 
Features,  moving  from  specific  knowledge-lean  features  to  more 
general  knowledge-rich  features,  was  helpful.  They  stated  it  was 
easier  to  assess  Major  Features  such  as  Fingerspelling  and  Numbers 
before  Vocabulary,  and  Grammar  and  Structuring  Space  were  easier 
to  assess  before  Interpreting.  They  did  make  one  suggestion; 
Structuring  Space  should  immediately  follow  Classifiers/SASSes 
rather  than  following  Grammar. 

Skills  and  Errors 


Initially  it  took  time  for  the  raters  to  locate  the  applicable 
skills  within  the  instrument  as  they  were  observed  in  the  sample 
interpretation.  For  example,  while  rating  the  first  interpretation, 
Rater  1  moved  back  and  forth  in  the  instrument,  seemingly  ignoring 
the  specific  order  in  which  the  Major  Features  were  listed.  At 
times,  she  spent  time  looking  for  specific  skills  when  the 
interpretation  warranted  a  comment.  Flowever,  when  rating  the 
second  interpretation,  Rater  1  followed  the  sequence  of  the  skills  as 
listed  in  the  instrument  more  closely,  with  much  less  apparantly 
random  movement.  In  contrast,  Rater  2,  followed  the  order  of  the 
instrument.  Often  she  previewed  the  next  several  pages  so  as  to 
know  the  next  skills  that  were  to  be  rated.  When  she  was  unable  to 
find  the  page  where  a  specific  comment  or  a  rating  should  be  placed, 
she  wrote  them  on  a  blank  piece  of  paper.  When  the  appropriate 
place  to  make  a  specific  comment  was  located,  she  then  copied  the 
corresponding  notes  from  the  extra  piece  of  paper  onto  the 
instrument. 
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Both  raters  commented  that  the  errors,  as  written  in  the 
instrument,  associated  with  the  skills  were  written  in  a  negative 
rather  than  a  positive  manner,  making  the  raters  feel  their 
assessment  was  more  negative  than  positive  in  approach.  This  can 
possibly  be  accounted  for  by  the  fact  that  educators  attempt  to 
highlight  the  positive  in  interpreters'  work.  Even  when  there  is 
negative  feedback  to  give  to  the  interpreters,  it  is  stated  in  a 
positive  manner.  In  the  development  of  the  instrument  each  skill 
was  partially  defined  by  the  possible  errors  that  interpreters  make 
when  demonstrating  particular  skills,  thus  causing  the  appearance 
of  negativity  in  the  instrument. 

Definitions 


Both  Raters  1  and  2  minimally  referred  to  the  definitions 
while  rating.  They  indicated  that  they  had  carefully  read  the  entire 
instrument,  including  the  definitions,  when  they  first  received  the 
materials,  and  both  indicated  that  they  knew  the  intent  of  each  skill 
and  possible  error.  They  remarked  that  if  they  were  unsure  of  the 
meaning  of  a  particular  item,  they  referred  to  the  examples  provided 
with  it.  They  indicated  that  the  examples  were  extremely  helpful  in 
reminding  them  what  to  assess. 

Length  of  Stimulus 

Both  raters  agreed  that  the  length  of  the  tape,  five  minutes, 
was  sufficient  to  assess  a  sample  of  an  interpreter's  work.  With 
respect  to  the  sample  tape  used  in  the  validation  process,  both 
raters  indicated  that  the  speaker  was  well  paced  and  that  the 
content  of  the  speech  was  conducive  to  viewing  samples  of  skills  in 
each  of  the  eight  Major  Features. 

They  both  predicted  that  the  time  required  to  rate  an 
interpretation  would  be  reduced  after  using  the  instrument  with  the 
same  audio  tape  stimulus  a  number  of  times. 
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The  Suggested  Procedure 

Following  the  researcher's  instructions,  each  rater  viewed  the 
interpretations  without  sound  at  least  once  during  the  assessment 
procedures.  Rater  1  viewed  the  interpretations  with  sound  several 
times  before  viewing  it  without  sound.  In  contrast,  Rater  2  elected 
to  first  view  the  interpretation  without  sound;  she  then  alternated 
between  viewing  with  and  without  sound  throughout  the  rating 
process. 

Rater  1  commented  that  she  did  not  want  to  spend  time  at  the 
beginning  listening  without  sound.  However,  she  suggested  that  if 
she  were  to  do  it  over  again,  she  would  first  begin  without  sound  and 
record,  on  a  separate  sheet  of  paper,  her  initial  reactions  to  the 
interpretation.  Then,  using  the  instrument,  she  would  rate  the 
interpretation  with  sound. 

The  4-Point  Rating  Scale 

Both  raters  used  the  4-point  rating  scale  for  the  skills 
without  difficulty.  However,  the  raters  used  the  scale  to  rate  errors 
differently.  Rater  1  marked  a  'V  when  there  were  only  1  or  2 
mistakes.  It  meant  that  the  error  was  noticeable,  but  nothing  more. 
Marking  a  '2'  meant  either  there  were  a  few  mistakes,  or  that  a 
single  error  was  important.  This  error  needed  to  be  addressed  and 
was  more  serious  than  the  errors  rated  T.  If  she  felt  that  the 
performance  was  very  weak  and  was  serious,  she  marked  '3'.  She 
used  '4'  if  the  entire  performance  of  that  skill  was  inaccurate. 

Rater  2  marked  'I1  when  a  few  errors  were  present.  When 
several  errors  were  present  but  the  interpretation  was  still 
"comfortable",  she  used  a  '2'.  When  comprehension  of  the  meaning 
was  impeded,  she  marked  a  '3'.  And  finally,  she  used  a  '4'  when  the 
performance  of  the  skill  was  inaccurate  or  when  understanding  was 
not  possible. 
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Revisions 
Major  Features 

The  sequence  of  the  Major  Features  was  changed  in  accordance 
with  the  suggestion  that  Structuring  Space  would  be  easier  to  rate 
if  it  followed  immediately  after  Classifiers/SASSes.  An 
interpretation  using  Classifiers/SASSes  requires  the  use  of  space  in 
a  specific  manner.  Therefore,  placing  Structuring  Space  after  this 
feature  made  good  sequential  sense.  No  other  changes  were  made  to 
the  sequence  of  the  Major  Features. 

Skills.  Errors,  and  Definitions 


Several  changes  were  made  to  the  skills,  errors,  and 
definitions.  The  number  of  changes  made  by  type  are  summarized  in 
Table  9.  As  shown,  the  greatest  number  of  changes  (41)  were  made 
within  the  Major  Feature  of  Interpreting.  Structuring  Space  had  the 
second  highest  number,  with  a  total  of  33  revisions.  The  least 
number  of  changes  occurred  in  Numbers  (2)  and  Vocabulary  (11). 

The  greatest  number  of  revisions  were  of  an  editorial  nature. 
Editorial  revisions  included  adding  or  changing  the  example  provided 
with  a  skill  or  error,  further  clarification,  and  improved  consistency 
in  wording,  punctuation,  and  spelling. 

After  Field  Test  1,  17  changes  in  sequence  occurred.  All  17 
were  sequence  changes  within  the  same  Major  Feature.  That  is,  no 
skills  or  errors  were  moved  to  a  different  Major  Feature;  they  were 
only  moved  within  the  Major  Feature  in  which  they  were  initially 
listed. 

Some  skills  and/or  errors  (26)  were  aggregated  into  one.  For 
example,  if  a  skill  was  redundant  and  did  not  provide  additional 
diagnostic  value,  then  it  was  aggregated  with  another.  Thus,  the 
two  skills  were  combined  into  one  skill  without  losing  any 
diagnostic  information. 
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Six  editorial  changes  were  made  to  the  last  section,  Rater's 
Impressions.  No  other  changes  were  made. 

Table  9 

Revisions  of  Skills.  Errors,  and  Definitions  within  Major  Features  as 

a  Result  of  Field  Test  1 


Major 

Feature 

Additions 

Deletions 

Sequence 

Changes 

Editorial 

Aggre- 

gated 

T  otals 

Finger¬ 

spelling 

0 

0 

0 

21 

0 

21 

Numbers 

0 

0 

0 

2 

0 

2 

Vocabulary 

0 

0 

3 

8 

0 

1  1 

Classifiers 

&  SASSes 

1 

2 

0 

1  6 

0 

1  9 

Grammar 

5 

0 

2 

1  0 

4 

21 

Structuring 

Space 

0 

0 

5 

1  6 

1  2 

33 

Interpreting 

2 

0 

5 

26 

8 

41 

Composure 

Appearance 

0 

0 

2 

1  1 

2 

1  5 

TOTALS 

8 

2 

1  7 

1  10 

26 

163 

Rating  Scale 

Based  on  the  different  ways  Rater  1  and  Rater  2  used  the 
rating  scale  for  the  errors  and  following  further  consultation  with 
them,  the  rating  scale  was  revised  as  follows: 
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NT  -  not  in  the  target  message 

0  -  no  errors  present 

1  -  low  frequency,  low  severity 

2  -  high  frequency,  low  severity 

3  -  low  frequency,  high  severity 

4  -  high  frequency,  high  severity. 

The  addition  of  scale  points  NT  and  0  allowed  raters  to  specify 
whether  or  not  a  skill  was  demonstrated  in  the  target  language.  NT 
was  used  to  specify  that  it  was  not  demonstrated  in  the  target 
language  and  0  was  used  when  it  was  demonstrated  in  the  target 
language  and  no  errors  occurred. 

To  provide  easy  reference  for  the  raters,  the  operational 
definitions  of  the  rating  scale  were  provided  in  the  revised 
instrument  at  the  beginning  of  each  Major  Feature. 

Additions  Made  to  the  Instrument 


To  assist  raters  in  locating  specific  skills  and  errors  a  Cross 
Reference  Index  was  developed  and  placed  at  the  back  of  the 
instrument.  The  Cross  Reference  Index  listed  specific  skills  and 
their  corresponding  number(s)  to  assist  the  raters  in  locating 
particular  skills. 

The  numbering  of  the  definitions  with  their  corresponding 
skills  and  errors  was  revised  so  that  it  was  consecutive  from  the 
beginning  to  the  end  of  the  instrument  as  opposed  to  each  Major 
Feature  starting  with  T.  To  assist  raters  in  locating  specific 
sections  in  the  interpretation  that  they  felt  would  elicit  certain 
skills,  a  transcript  of  the  audio  tape  stimulus  was  written. 

After  viewing  raters  actually  use  the  instrument,  it  became 
apparent  that  the  space  provided  for  comments  and  feedback  was 
insufficient.  Therefore,  additional  space  was  provided. 

Because  of  the  importance  of  viewing  the  interpretation 
without  sound,  a  reminder  was  added  at  the  beginning  of  each  Major 
Feature  for  the  rater  to  view  the  interpretation  at  least  twice 
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without  sound.  Both  Rater  1  and  Rater  2  indicated  that  viewing  the 
interpretation  only  once  was  not  sufficient.  Thus,  the  requirement 
to  view  it  at  least  twice  without  sound  was  added  and  placed  on  the 
same  page  as  the  operational  definitions  for  the  rating  scale. 

Results  of  the  Revisions 

The  third  draft  of  the  instrument  was  spiral  bound  so  that  it 
would  lay  flat,  thereby  permitting  the  rater  to  assess  the 
interpretation  without  having  to  hold  it  open.  Also,  by  laying  the 
instrument  flat,  the  rater  could  simultaneously  see  the  definitions 
on  the  left  page  and  the  corresponding  skills  and  possible  errors  on 
the  right  page.  Each  Major  Feature,  Rater's  Impressions,  and  the 
Cross  Reference  Index  were  printed  on  different  colors  of  paper  to 
help  distinguish  one  section  from  another. 

Preparation  of  the  Instruction  Manual 

Following  Field  Test  1,  an  Instruction  Manual  was  developed  to 
accompany  the  revised  instrument.  The  Manual  contained 
information  on  the  instrument's  purpose  and  goals  along  with  its 
limitations.  The  organization  of  the  instrument  was  described  as 
well  as  the  terminology  used.  For  example,  because  ASL  is  not  a 
written  language,  the  writing  of  ASL  was  discussed  and  examples 
were  provided  to  clarify  the  meaning  of  the  writing.  The 
qualifications  of  the  user,  the  rater,  were  described  and  the 
administration  procedures  and  the  rating  process  were  explained. 

Eleven  sections  were  included  in  the  manual:  Purpose  and 
Goals,  Terminology,  Limitations,  Assumptions,  Rater  Qualifications, 
Instrument  Format,  The  Stimulus,  Administration  Procedures,  Rating 
Process,  Results,  and  References. 
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CHAPTER  5:  FIELD  TEST  2 

Following  Field  Test  1  and  the  revisions  made  to  the 
instrument,  a  second  field  test  was  conducted  with  raters  from 
California.  The  raters  used  the  third  draft  of  the  instrument  with 
the  instruction  manual  to  rate  two  five-minute  video  taped 
interpretation  samples.  The  methods  used  for  and  the  results  from 
Field  Test  2  are  described  and  discussed  in  the  balance  of  this 
chapter. 

Method 

The  Audio  Tape  Stimulus 

The  English  audio  tape  stimulus  used  for  Field  Test  2  was  the 
same  as  that  used  in  Field  Test  1.  No  changes  were  made  to  the 
stimulus  audio  tape. 


The  Interpreters 

A  change  in  interpreters  was  made  for  Field  Test  2.  The  non- 
certified,  experienced  interpreter  used  in  Field  Test  1  was  replaced 
by  a  certified,  experienced  interpreter  for  Field  Test  2.  The  latter 
interpreter  held  certification  from  both  national  interpreter 
organizations  in  North  America,  AVLIC  and  RID.  Thus,  external 
validation  of  the  interpreter's  interpretation  ability  was 
acknowledged  by  two  national  certifying  bodies  and  it  could  be  said 
this  interpreter  was  an  expert.  This  interpreter  was  called 
Interpreter  3  to  avoid  confusion  with  Field  Test  1. 

Interpreter  3  had  10  years  of  interpreting  experience.  She 
freelanced  20  hours  a  week  primarily  in  mental  health,  legal,  and 
educational  (K- 1 2)  settings.  Annually,  she  attended  between  150- 
170  hours  of  workshops  related  to  interpreting. 

Interpreter  3  learned  ASL  through  both  hearing  and  Deaf  people. 
She  had  a  Deaf  family  member  and  was  not  a  graduate  of  an 
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interpreting  program.  She  held  a  university  degree  in  a  field 
different  from  interpreting. 

She  was  a  female,  native  English  speaker  living  in  Alberta  and 
was  a  member  of  both  the  local  and  national  interpreter 
organizations. 


Developing  the  Video  Tape 

Interpreter  3  was  video  taped  following  the  same  procedures 
as  those  used  in  Field  Test  1. 

Using  the  Instrument  and  Rating  Performances 

The  Raters 


To  locate  a  group  of  raters  who  had  no  prior  involvement  in  the 
developmental  stages  of  the  instrument,  raters  were  selected  from 
the  United  States.  This  selection  took  place  because  the  number  of 
qualified  Canadians  without  prior  involvement  with  the  previous 
development  and  validation  stages  of  the  instrument  was 
insufficient. 

A  location  in  California  was  selected  that  had  a  group  of 
raters  qualified  to  assist  in  Field  Test  2.  Ten  raters,  living  in  the 
same  geographic  region,  were  contacted  by  phone  and  asked  if  they 
were  willing  to  participate  in  a  field  study  to  further  validate  a 
newly  developed  ASL/English  diagnostic  assessment  instrument. 

The  raters  were  chosen  due  to  their  expertise  and  experience  in 
English/ASL  interpretation  and  interpreter  assessment  either  in  the 
classroom  and/or  on  the  job.  All  had  experience  assessing 
interpreters  at  various  skill  levels.  Other  than  one,  who  had  limited 
exposure  to  portions  of  the  instrument  in  an  editorial  capacity,  none 
of  the  raters  were  involved  in  the  developmental  stages  of  the 
instrument. 
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Of  the  ten  raters  asked,  nine  agreed  to  participate. 
Consequently,  one  person  indicated  he  could  not  complete  the  task 
due  to  the  lack  of  time  he  had  available.  Of  the  remaining  eight,  one 
was  unable  to  complete  the  task  due  to  health  reasons,  although  she 
was  able  to  review  the  instrument  and  instruction  manual  providing 
written  and  oral  comments  in  a  meeting  held  with  the  researcher. 
The  seven  remaining  raters  were  able  to  complete  the  task. 

A  summary  of  the  qualifications  of  the  seven  raters  who 
completed  the  task  is  provided  in  Table  10. 

All  of  the  seven  raters  were  members  of  and  held  national 
certification  from  RID  (Registry  of  Interpreters  for  the  Deaf).  Five 
held  the  Comprehensive  Skills  Certificate  (CSC)  and  two  held  the 
Master  Comprehensive  Skills  Certificate  (MCSC)  which  RID  offered 
for  a  short  period  of  time  to  those  who  had  already  received  a  CSC 
and  were  tested  again. 

Table  10 

Field  Test  2  Rater  Qualifications 

Qualific¬ 
ations  Raters _ 


1 

2 

3 

4 

5 

6 

7 

Certified 

CSC 

CSC 

CSC 

CSC 

MCSC 

CSC 

MCSC 

Years 

Evaluat¬ 

ing 

interps 

1  0 

6 

1  2 

1  3 

20 

5 

1  8 

Years 

teaching 

1  0 

0 

1  3 

1  3 

20 

5 

1  8 

Native 

ASL 

No 

No 

No 

No 

No 

No 

Yes 

Native 

English 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 
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All  seven  had  experience  evaluating  interpreters.  This 
experience  ranged  from  5  to  20  years  with  a  mean  of  12  years.  Six 
of  the  seven  raters  had  experience  teaching  interpreters  in  an 
educational  environment.  This  experience  ranged  from  5  to  20  years 
with  a  mean  of  13.2  years.  All  six  taught  both  interpreting  students 
and  employed  interpreters  and  all  except  one  were  currently 
teaching.  The  one  rater  with  no  teaching  experience  worked 
primarily  as  an  evaluator  of  employed  interpreters  on  a  regular 
basis,  but  did  not  provide  formal  education  to  them. 

Only  one  of  the  seven  identified  herself  as  a  native  ASL  user 
and  all  seven  were  native  English  speakers.  All  seven  raters  were 
female  ranging  in  age  from  31  to  65  years,  with  a  mean  of  45. 

Procedures  for  Rating  Performances 

Due  to  prior  time  commitments,  two  of  the  seven  raters  who 
completed  the  task  were  unable  to  attend  the  scheduled  meeting. 
Consequently,  separate  meetings  were  held  with  these  two  raters; 
the  researcher  met  with  the  remaining  five  in  a  group. 

Group  Meeting 

The  five  raters  received  a  copy  of  the  instrument  and 
instruction  manual  two  weeks  in  advance  of  the  researcher's  arrival 
to  California.  They  were  asked  to  read  the  materials  and  make 
written  comments  prior  to  the  scheduled  group  meeting  with  the 
researcher.  All  were  told  that  discussions  about  the  instrument  and 
the  instruction  manual  would  be  held  at  the  conclusion  of  the 
assessments.  Thus,  all  seven  raters  had  to  depend  on  the  instruction 
manual  for  the  information  they  needed  to  conduct  the  assessments. 

The  meeting  began  with  each  of  the  five  raters  completing  the 
assessment  of  one  of  the  two  interpretation  samples  in  an  assigned 
random  order.  Each  had  her  own  video  playback  unit,  monitor,  and 
video  tape.  Four  of  the  five  machines  had  a  remote  control.  They 
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were  each  given  a  blank  instrument  to  mark  as  well  as  a  written 
English  transcript  of  the  interpreted  lecture.  They  were  asked  to 
record  any  comments  about  the  instrument  and  rating  procedures  on 
the  instrument  and  the  instruction  manual  mailed  earlier  to  them. 
Each  worked  independently. 

After  all  of  the  five  raters  had  completed  rating  the  first 
interpretation  and  after  a  lunch  break,  a  discussion  was  held  on  the 
content  and  the  utility  of  the  instrument.  Because  the  discussion 
lasted  the  rest  of  the  afternoon,  it  was  not  possible  to  complete  the 
assessment  of  the  second  interpretation  sample  in  the  group  setting. 
The  five  raters  agreed  to  complete  the  second  rating  on  their  own 
time  and  mail  the  video  tape,  completed  instruments,  and  manual 
back  to  the  researcher. 

Individual  Meetings 

As  with  the  raters  who  attended  the  scheduled  group  meeting, 
the  two  raters  unable  to  attend  the  meeting  received  a  copy  of  the 
instrument  and  instruction  manual.  In  addition,  they  also  received 
the  two  video  tape  interpretations,  the  written  English  transcript  of 
the  lecture  corresponding  to  the  interpretations,  and  two 
instruments  to  be  used  for  the  assessments. 

The  researcher  met  with  the  two  raters  individually.  One 
completed  assessing  both  interpretations;  the  other  assessed  one 
interpretation  sample.  On  each  occasion,  the  raters  discussed  the 
instrument  and  the  rating  process  with  the  researcher. 

Collecting  Raters1  Reactions  to  the  Assessment  Procedures 

The  researcher  orally  asked  the  raters  questions,  five  in  the 
group  meeting  and  two  individually,  about  their  reactions  to  the 
assessment  instrument  and  rating  procedures.  First,  they  were 
asked  to  provide  general  comments.  Then  they  were  asked  more 
specific  questions  on  the  content,  the  utility,  and  the  flow  of  the 


instrument.  The  questions  asked  were  the  same  as  the  questions 
asked  in  Field  Test  1  with  the  addition  of  questions  regarding  the 
instruction  manual. 

In  addition  to  the  responses  to  these  questions,  the  researcher 
took  notes  during  the  assessments  conducted  in  the  group  setting. 

Of  particular  importance  was  the  way  in  which  the  raters  worked 
through  the  instrument. 


Findings 

Time  Required  to  Complete  the  Assessments 

Table  11  summarizes  the  time  it  took  each  rater  to  complete 
the  rating  for  each  sample  interpretation,  along  with  which  sample 
interpretation  was  rated  first  and  which  was  rated  second.  As 
indicated  earlier,  five  raters,  2-5,  completed  their  first  assessment 
in  the  presence  of  the  researcher  and  their  second  one  while  alone. 
Raters  1  and  7  completed  both  of  the  assessments  without  the 
researcher  being  present. 

The  mean  for  the  length  of  time  required  to  complete  the 
assessment  for  Interpretation  2,  rated  first  by  Raters  1-4,  was  2 
hours  and  8.75  minutes  with  a  standard  deviation  of  41.29  minutes. 
The  mean  for  Interpretation  3,  which  was  rated  first  by  Raters  5-7, 
was  2  hours  with  a  standard  deviation  of  0. 

When  assessing  their  second  interpretation  sample,  the  mean 
was  2  hours  with  a  standard  deviation  of  0  for  Interpretation  2  and  a 
mean  of  1  hour  and  52.5  minutes  and  a  standard  deviation  of  33.45 
minutes  for  Interpretation  3.  Although  there  was  more  variance 
among  the  raters  when  assessing  Interpretation  3  than 
Interpretation  2,  the  length  of  time  required  to  complete  the 
assessment  was  less. 
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Table  11 

Time  Required  to  Rate  Interpretation  Samples  2  and  3  across  Seven 

Raters 


Interpre - 
tation  Raters 


1 

2 

3 

4 

5 

6 

7 

Mean 

SD 

2  rated 

first 

1  hr. 

10  min. 

3  hrs. 

5  min. 

2  hrs. 

20  min. 

2  hrs. 

2  hrs. 

8.75 

min. 

41.29 

min. 

3  rated 

first 

2  hrs. 

2  hrs. 

2  hrs. 

2  hrs. 

0 

2  rated 

second 

2  hrs. 

2  hrs. 

2  hrs. 

2  hrs. 

0 

3  rated 

second 

1  hr. 

30 

min. 

2  hrs. 

50 

min. 

1  hr. 

40 

min. 

1  hr. 

30 

min. 

1  hr. 

52.5 

min. 

33.45 

min. 

Differ¬ 

ence 

between 

first  and 

second 

rating 

20 

min. 

more 

1  5 

min. 

less 

40 

min. 

less 

30 

min. 

less 

no 

change 

no 

change 

no 

change 

Three  raters,  2,  3,  and  4,  who  first  rated  Interpretation  2 
followed  by  Interpretation  3,  completed  their  second  assessment  in 
15  to  40  minutes  less  time  than  it  took  them  to  complete  their  first 
assessment.  Rater  1,  rating  the  interpretations  in  the  same  order, 
took  20  minutes  longer  for  the  second  assessment. 

The  three  raters,  5,  6,  and  7,  who  first  rated  Interpretation  3 
followed  by  Interpretation  2,  required  the  same  amount  of  time  (2 
hours)  to  rate  each  interpretation  sample. 
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Based  on  the  researcher's  observations  and  the  comments  of 
the  raters,  several  plausible  explanations  account  for  the 
differences  in  the  amount  of  time  required  to  rate  the  interpretation 
samples.  Rater  1,  who  took  longer  to  complete  the  second 
assessment,  hurried  during  the  first  assessment  to  meet  with  the 
researcher  at  a  predetermined  time.  This  could  explain  why  it  took 
Rater  1  longer  the  second  time. 

Two  possible  reasons  for  the  decrease  in  time  are  practice  and 
the  sequence  of  rating  the  samples.  All  seven  raters  stated  that  the 
rating  task  was  easier  the  second  time,  because  they  knew  the 
instrument  and  where  skills  were  located  better,  thereby  reducing 
their  reliance  on  the  Cross  Reference  Index. 

Raters  1,  2,  3,  and  4  rated  interpretation  2,  the  novice, 
followed  by  interpretation  3,  the  expert.  Three  of  the  four  took  less 
time.  In  contrast,  Raters  5,  6,  and  7  rated  the  expert  first  with  no 
change  in  the  time  it  required  to  rate  the  novice.  One  possible 
explanation  of  these  findings  is  that  less  time  is  required  to  rate  an 
expert  than  a  novice.  However,  practice  also  must  be  taken  into 
account.  Although  these  raters  had  practice  with  the  instrument,  it 
did  not  take  them  less  time.  Even  with  practice  it  may  not  be 
possible  to  rate  a  novice  interpreter  in  less  time  than  an  expert. 

Rating  Consistency  Within  Raters 

As  part  of  the  process  for  determining  the  utility  of  the 
instrument,  rating  consistency  between  skill  ratings  and 
corresponding  error  ratings  within  each  of  the  seven  raters  was 
analyzed.  The  findings  showed  that  there  was  slightly  less  than 
100%  consistency  within  each  rater.  That  is,  when  the  skill  was 
rated  'No',  meaning  the  skill  was  not  performed  accurately,  then  at 
least  one  of  the  subsequent  errors  was  rated  between  1  and  4.  When 
the  skill  was  rated  'Yes',  meaning  the  skill  was  performed 
accurately,  then  in  all  of  the  subsequent  related  errors  either  'O'  or 
'NT'  was  marked;  there  were  no  errors  present.  When  NT,  not  in  the 


target  message,  was  marked  for  the  skill,  then  in  all  of  the 
subsequent  related  errors  'NT'  was  marked. 
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Agreement  Among  Raters  Across  Skills  Within  Each  Major  Feature 

The  agreement  among  raters  on  the  skills  within  each  Major 
Feature  for  Interpretation  2  and  Interpretation  3  is  described  in  this 
section. 

Interpretation  2 

Table  12  summarizes  the  agreement  among  raters  across  the 
skills  within  each  Major  Feature.  The  table  indicates  the  number  of 
skills  in  each  Major  Feature,  how  many  times  at  least  five  raters 
agreed  on  their  rating  for  the  skills,  and  the  percent  of  skills  for 
which  they  agreed  on  their  rating. 

At  the  skill  level  for  Interpretation  2,  the  Major  Feature  of 
Vocabulary  with  four  skills  had  the  highest  agreement.  At  least  five 
of  the  raters  agreed  on  each  of  the  four  skills  included  within 
Vocabulary.  Good  agreement  was  present  in  three  additional  Major 
Features:  Fingerspelling  with  80%,  Classifiers/SASSes  with  71%, 
and  Structuring  Space  with  71%. 

The  lowest  agreement  was  in  the  Major  Feature  of  Numbers 
with  43%  agreement.  Interpreting  and  Composure  and  Appearance 
had  50%  agreement  with  Grammar  having  slightly  more  agreement  at 
66%. 

With  the  exception  of  Numbers,  the  rating  of  the  skills  in  the 
first  five  Major  Features  had  higher  agreement  than  the  latter  three 
Major  Features.  One  possible  reason  for  this  is  the  difference 
between  the  knowledge-lean  and  knowledge-rich  Major  Features. 
Perhaps  it  is  easier  to  agree  on  the  knowledge-lean  features, 
requiring  assessment  of  isolated  skills,  than  the  knowledge-rich 
features,  requiring  assessments  of  skills  occurring  throughout  the 
entire  interpretation. 


Table  12 

Agreement  Among  Raters  for  Interpretation  2  across  Skills  within 

Each  Major  Feature 
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Major  Features 

No.  of  skills 

No.  of  skills 

at  least  5 

raters 

aqreed 

Percent  of 

agreement  of 

at  least  5 

raters 

Finqerspellinq 

1  0 

8 

80 

Numbers 

7 

3 

43 

Vocabulary 

4 

4 

1  00 

Classifiers/ 

SASSes 

7 

5 

7  1 

Structuring 

Space 

7 

5 

7  1 

Grammar 

6 

4 

66 

Interpretinq 

1  4 

7 

50 

Composure  and 

Appearance 

4 

2 

50 

TOTALS 

59 

38 

64 

Interpretation  3 

The  agreement  among  raters  on  the  skills  in  each  Major 
Feature  for  Interpretation  3  is  summarized  in  Table  13.  At  the  skill 
level  for  Interpretation  3,  the  Major  Feature  of  Composure  and 
Appearance  had  the  highest  agreement  at  100%.  Good  agreement  was 
present  in  one  additional  Major  Feature:  Vocabulary  with  75%.  The 
lowest  agreement,  29%  occurred  in  both  Classifiers/SASSes  and 
Structuring  Space.  The  agreement  for  the  remaining  four  Major 
Features  ranged  from  50  to  64%. 


Table  13 

Agreement  Among  Raters  for  Interpretation  3  across  Skills  within 

Each  Major  Feature 
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Major  Features 

No.  of  skills 

No.  of  skills 

at  least  5 

raters 

agreed 

Percent  of 

agreement  of 

at  least  5 

raters 

Finqerspelling 

1  0 

5 

50 

Numbers 

7 

4 

57 

Vocabulary 

4 

3 

75 

Classifiers/ 

SASSes 

7 

2 

29 

Structuring 

Space 

7 

2 

29 

Grammar 

6 

3 

50 

Interpreting 

1  4 

9 

64 

Composure  and 

Appearance 

4 

4 

100 

TOTALS 

59 

32 

54 

Agreement  Among  Raters  Across  Errors  within  Each  Major  Feature 

As  a  result  of  the  raters'  comments  in  Field  Test  1  and  after 
checking  back  with  the  raters  in  Field  Test  2,  the  assessment 
ratings  at  the  error  level  were  aggregated  to  determine  differences 
between  1)  errors  either  not  present  or  negligible,  2)  frequent 
occurrence  of  errors  but  not  severe  errors,  and  3)  severe  errors.  The 
following  three  groupings  were  formed: 

a)  NT,  O,  1 

b)  2 

c)  3,  4. 
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In  group  (a)  either  no  errors  were  present,  'O'  or  'NT',  or  the  error 
was  negligible  with  a  rating  of  '1'.  In  group  (b)  the  error  occurred 
frequently  but  was  not  severe  in  terms  of  the  potential  for 
misunderstandings.  In  group  (c)  the  error  was  severe,  either 
occurring  occasionally  or  frequently. 

Interpretation  2 

The  agreement  among  raters  on  the  assessment  rating  of  the 
errors  across  Major  Features  in  Interpretation  2  is  illustrated  in 
Table  14.  Changes  in  percent  of  agreement  occurred  from  the  skill 
level  to  the  error  level  on  Interpretation  2.  Agreement  for 
Classifiers,  Interpreting,  and  Composure  and  Appearance  from  the 
skill  level  of  71%,  50%,  and  50%  was  increased  at  the  error  level  to 
88%,  76%  and  88%,  respectively.  Numbers  had  the  most  significant 
increase  which  was  from  43%  at  the  skill  level  to  100%  at  the  error 
level. 

There  was  a  decrease  in  agreement  from  Vocabulary, 
Structuring  Space,  and  Grammar  at  the  skill  level  of  100%,  71%  and 
66%,  to  the  error  level  of  90%,  46%  and  56%,  respectively. 

The  Major  Feature  of  Fingerspelling  remained  the  same  at  80% 
for  both  the  skill  level  and  the  error  level. 

The  first  four  Major  Features,  the  more  knowledge-lean 
features,  had  more  agreement  than  three  of  the  four  knowledge-rich 
features.  This  is  consistent  with  the  rating  at  the  skill  level  for 
Interpretation  2. 

Also,  the  agreement  overall  for  Interpretation  2  is  higher  at 
the  error  level  (77%)  when  compared  to  the  rating  of  the  skill  level 
(64%).  This  may  indicate  that  reporting  agreement  at  the  skill  level 
is  too  gross  of  a  measurement  and  that  reporting  at  the  error  level 
is  more  appropriate  and  informative. 
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Table  14 

Rater  Agreement  on  Interpretation  2  across  Errors  at  the  Aggregated 

Rating  Level  within  Each  Major  Feature 


Major  Features 

No.  of  errors 

No.  of  errors 

at  least  5 

raters 

agreed 

Percent  of 

agreement  of 

at  least  5 

raters 

Finqerspelling 

20 

1  6 

80 

Numbers 

1  9 

1  9 

1  00 

Vocabulary 

20 

1  8 

90 

Classifiers/ 

SASSes 

1  7 

1  5 

88 

Structuring 

Space 

24 

1  1 

46 

Grammar 

1  8 

1  0 

56 

Interpreting 

4  2 

32 

76 

Composure  and 

Appearance 

1  7 

1  5 

88 

TOTALS 

1  77 

1  36 

77 

Interpretation  3 

As  indicated  in  Table  15,  the  agreement  across  Major  Features 
at  the  error  level  was  very  good.  Four  of  the  eight  features  had 
100%  agreement  with  the  remaining  four  ranging  in  agreement  from 
88  -  95%.  Again  the  first  four  Major  Features  had  higher  agreement 
than  the  latter,  more  knowledge-rich  features,  with  the  exception  of 
Composure  and  Appearance. 

The  overall  agreement  at  the  skill  level  among  raters  for 
Interpretation  3  (54%)  was  less  than  Interpretation  2  (64%).  This  is 
a  surprising  outcome.  A  certified  expert  interpreter  would  likely 
have  less  errors  than  a  less  experienced  non-certified  interpreter 
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and  therefore  the  rating  should  be  more  consistent.  Or  perhaps, 
errors  are  committed  but  they  are  more  subtle,  thus  making  it 
difficult  to  discern  them. 

Table  15 

Rater  Agreement  on  Interpretation  3  across  Errors  at  the  Aggregated 

Rating  Level  within  Each  Major  Feature 


Major  Features 

No.  of  errors 

No.  of  errors 

at  least  5 

raters 

agreed 

Percent  of 

agreement  of 

at  least  5 

raters 

Finqerspelling 

20 

20 

1  00 

Numbers 

1  9 

1  9 

1  00 

Vocabulary 

20 

1  9 

95 

Classifiers/ 

SASSes 

1  7 

1  7 

1  00 

Structuring 

Space 

24 

2  1 

88 

Grammar 

1  8 

1  6 

89 

Interpreting 

42 

39 

93 

Composure  and 

Appearance 

1  7 

1  7 

100 

TOTALS 

1  77 

1  68 

95 

As  with  the  Interpretation  2,  this  result  indicates  that  the 
interpretation  results  are  more  meaningful  if  reported  at  the  error 
level  instead  of  the  skill  level. 

There  was  much  higher  overall  agreement  at  the  error  level  on 
Interpretation  3  (95%)  compared  to  the  agreement  on  Interpretation 
2  (77%).  This  indicates  that  when  assessing  an  interpretation 
performed  by  an  expert  certified  interpreter,  there  is  higher  rater 
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agreement  at  the  error  level  than  when  assessing  an  interpretation 
performed  by  a  less  experienced,  non-certified  interpreter. 

Revisions 


Skills.  Errors,  and  Definitions 

Using  the  agreement  among  raters  results  and  the  written  and 
oral  comments  provided  by  the  seven  raters,  the  instrument  was 
revised  as  follows.  The  primary  revisions  were  to  clarify  the 
writing  and  to  increase  the  writing  consistency  within  the  skills, 
errors,  and  definitions.  A  total  of  174  revisions  of  this  type  were 
made.  Table  16  provides  a  summary  of  the  changes  made  to  the 
instrument  within  each  Major  Feature  as  a  result  of  Field  Test  2. 

Table  16 

Revisions  of  Skills.  Errors,  and  Definitions  Following  Field  Test  2 


Major  Features 

Writing 

Changes 

Sequence 

Changes 

Additions 

Deletions 

Total 

Finqerspellinq 

2  1 

4 

3 

0 

28 

Numbers 

2  6 

0 

0 

0 

26 

Vocabulary 

1  5 

0 

0 

1 

1  6 

Classifiers/ 

SASSes 

2  0 

0 

0 

0 

20 

Structuring 

Space 

27 

0 

1 

0 

2  8 

Grammar 

1  5 

0 

0 

0 

1  5 

Interpretinq 

38 

0 

0 

0 

38 

Composure  and 

Appearance 

1  2 

0 

2 

0 

1  4 

TOTALS 

1  74 

4 

6 

1 

1  85 
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Four  sequence  changes  were  made,  all  within  the  Major  Feature 
of  Fingerspelling.  For  two  skills,  two  errors  were  reversed  to 
correspond  with  the  order  of  the  information  presented  in  the 
definition  of  the  skill. 

Six  new  errors  were  added;  one  error  was  deleted  because  it 
was  redundant  and  did  not  provide  any  additional  diagnostic 
information  that  was  not  already  present  elsewhere  in  the 
instrument. 

Within  Rater's  Impressions,  4  editorial  changes  were  made  to 
encourage  the  rater  to  write  clear  and  specific  feedback  to  the 
interpreter. 

Rating  Scale 

The  rating  scale  for  the  skills  was  altered  in  several  ways. 
"NT",  not  in  target  language  message,  ASL,  was  confusing  or  unclear 
to  five  of  the  raters.  Therefore,  this  was  changed  to  "NA,  not 
applicable,  not  in  the  target  language,  ASL".  "Y",  "yes"  was  further 
operationalized  to  "Yes,  the  skill  was  performed  accurately  100%  of 
the  time".  Similarly,  "N",  "no",  was  also  operationalized  further  to 
"No,  the  skill  was  not  performed  accurately  100%  of  the  time".  Only 
one  change  was  made  to  the  labels  used  to  identify  the  points  on  the 
5-point  scale;  "0,  no  errors  present"  was  changed  to  "no  errors 
observed". 

The  Instrument 


Within  the  instrument  itself,  a  'Reminder  Page'  was  added  to 
remind  raters  about  the  rating  process.  For  example,  the  skill 
"fingerspelling  is  accurately  used  for  emphasis"  had  three  errors:  1) 
eye  gaze  is  inaccurate,  2)  rhythm  or  cadence  is  inaccurate,  and  3)  it 
is  overused.  Three  of  the  seven  raters  rated  these  errors  unrelated 
to  the  skill  but  related  to  the  overall  fingerspelling  demonstrated  in 
the  interpretation  sample.  To  avoid  this  occurring  again  and  to 
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remind  raters  about  other  aspects  of  rating,  four  reminder  pages 
were  inserted  throughout  the  instrument. 

The  Table  of  Contents  was  changed  to  enhance  the  usability  of 
the  instrument.  Instead  of  only  identifying  the  page  number  for  each 
section,  the  colors  of  the  pages  were  provided  as  well,  thus,  making 
it  easier  to  locate  the  appropriate  section. 

The  Cross  Reference  Index  was  improved  by  adding  topics  to 
refer  raters  to  the  appropriate  location  in  the  index.  For  example,  if 
raters  looked  up  'space'  in  the  draft  used  for  Field  Test  2,  they  would 
be  unable  to  locate  the  page  number.  In  the  final  draft  a  topic  was 
added,  'space,  see  structuring  space'.  In  this  way  raters  could  more 
easily  locate  specific  skills  within  the  instrument  without  having  to 
look  up  three  of  four  possible  terms.  A  copy  of  the  revised 
instrument,  the  fourth  draft,  is  provided  in  Appendix  F. 

The  Instruction  Manual 


Two  sections  were  added  to  the  manual.  The  first  was  a  brief 
description  of  the  instrument  which  was  placed  after  the  purpose 
and  goals.  Thus,  raters  could  quickly  determine  whether  this  tool 
satisfied  their  needs.  The  second  section  was  a  description  of  the 
person  undergoing  the  assessment,  the  'interpreter  qualifications'. 
This  section  was  placed  after  the  description  of  the  rater 
qualifications. 

Some  headings  within  the  manual  were  altered  to  more  clearly 
describe  the  content  of  each  section.  For  example,  the  section 
called,  'Assumptions'  was  changed  to  'Assumptions  and  Prior 
Conditions'  and  'Stimulus'  was  changed  to  'Using  and  Developing 
Stimuli'. 

The  section  'Administration  Procedures'  was  subdivided  into 
several  subsections:  Amount  of  Time  Required  to  Rate  an 
Interpretation,  Requirements  Prior  to  Rating,  and  Viewing  the  Video 
Tape. 
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Editorial  changes  were  made  as  well.  In  addition  to  making  the 
writing  more  clear,  the  most  significant  change  was  changing  the 
term  'evaluatee'  to  'interpreter'  to  more  accurately  describe  the 
person  undergoing  the  assessment.  A  copy  of  the  second  draft  of  the 
manual  is  provided  in  Appendix  G. 
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CHAPTER  6:  SUMMARY  AND  CONCLUSIONS 
Purpose  of  the  Study 

In  many  situations,  laws  in  Canada  and  the  United  States 
require  the  provision  of  interpreters  to  allow  Deaf  people  equal 
access  to  the  community  in  which  they  live.  Interpreters  interpret 
for  Deaf  and  hearing  people  in  an  array  of  situations,  such  as  an 
appointment  with  a  doctor  or  a  lawyer,  a  meeting  with  a  supervisor 
or  employee,  or  a  lecture  on  prenatal  care  or  on  death  and  dying. 

In  the  United  States  and  Canada,  national  associations  of 
American  Sign  Language  (ASL)/English  interpreters  were 
incorporated  in  1972  and  1981,  respectively.  Not  only  are  national 
interpreting  organizations  relatively  new,  but  ASL/English 
interpreting  education  is  new  as  well.  This  being  the  case,  curricula 
taught  in  ASL/English  interpreting  programs  are  not  standardized 
and  are  based  on  the  expertise,  experience,  and  knowledge  of 
individual  instructors  and  on  individual  program  philosophies. 

Current  ASL/English  interpreting  curricula  focus  on 
interpreting  in  specific  settings,  for  instance,  business  and 
industry,  medical  and  mental  health,  and  with  specific  consumers 
such  as  deaf-blind,  oral  (those  who  depend  on  speechreading  rather 
than  sign  language),  and  deaf  individuals  with  minimal  language 
competencies.  The  curricula  fail  to  focus  on  the  identification  and 
acquisition  of  requisite  skills  required  to  render  competent 
ASL/English  interpretations.  Instead,  they  focus  on  acquiring  skills 
and  knowledge  related  to  working  in  various  settings  and  working 
with  various  consumers. 

In  the  field  of  ASL/English  interpretation,  certification 
performance  examinations  are  available  in  Canada  and  the  United 
States  through  the  Association  of  Visual  Language  Interpreters  of 
Canada  and  the  Registry  of  Interpreters  for  the  Deaf,  respectively. 
Both  of  these  examinations  are  proficiency-based  and  are  not 
diagnostic.  Therefore,  interpreters  taking  these  examinations  only 
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receive  information  of  their  success  or  failure.  No  information 
about  interpreters'  strengths  or  weaknesses  is  provided. 

Deaf  and  hearing  consumers  and  employers  of  interpreters  are 
in  agreement  that  most  graduates  of  ASL/English  interpreting 
programs  lack  the  necessary  skills  to  interpret  competently  upon 
graduation  (Disabled  Persons  Participation  Program,  1991).  This  is 
confirmed  by  the  observation  that  interpreters  who  took  the  AVLIC 
performance  exam  in  1990  had  a  failure  rate  of  79%.  Interpreters 
who  took  the  RID  performance  exam,  for  ASL/English  interpretation, 
in  fiscal  year  1990-1991,  and  received  their  results,  had  a  failure 
rate  of  approximately  54%.  Interpreters  who  took  the  exam  in  1991- 
1992,  and  received  their  results,  had  a  failure  rate  of  approximately 
27%. 

Both  curricula  and  national  certification  examinations  lack  the 
detailed  identification  of  the  skills  necessary  to  perform  a 
competent  interpretation  of  English  to  ASL.  Thus,  this  research  has 
focused  on  the  identification  of  skills  required  to  render  a 
competent  interpretation,  specifically,  interpreting  from  English  to 
ASL.  The  research  included  the  identification  and  definition  of 
skills  and  the  development  of  a  diagnostic  assessment  instrument  to 
assess  interpreters  with  various  skill  levels  and  experiences. 

Summary  of  Procedures 

A  systematic  approach  to  instrument  development  as  outlined 
by  authors  Crocker  and  Algina  (1986)  was  followed  to  develop  the 
instrument.  The  first  seven  of  the  ten  steps  outlined  by  Crocker  and 
Algina  (pp.  66-84)  were  followed.  These  were:  1)  determine  the 
purposes  for  which  the  assessment  scores  derived  from  the 
instrument  would  be  used,  2)  identify  and  describe  the  behaviors 
indicative  of  good  interpretation  practice,  3)  create  the  test 
specifications  and  the  method  of  recording  observed  behaviors,  4) 
construct  the  initial  pool  of  items,  5)  review  the  items  by  a  panel  of 
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interpretation  experts,  6)  field  test  the  instrument  using 
interpreter  educators,  and  7)  analyze  the  data. 

Step  1  to  Step  4:  Developing  the  Instrument 

The  primary  purpose  of  the  assessment  scores  derived  from 
the  instrument  was  to  provide  interpreters  with  various  levels  of 
skill  and  experience  with  a  detailed  diagnostic  assessment  of  their 
English-to-ASL  interpretation  skills. 

The  instrument  was  developed  by  first  identifying  skills 
generally  seen  in  English-to-ASL  interpretations.  Identification 
was  accomplished  by  reviewing  the  related  literature,  both  print  and 
video,  and  conducting  observational  or  task  analysis  of  interpreters 
with  various  skill  levels  and  experience.  All  observations  were 
conducted  by  the  researcher. 

Skills  were  defined  as  interpretation  elements  required  to 
provide  equivalency  of  meaning  from  the  source  language  of  English 
to  the  target  language  of  ASL.  The  result  was  a  compilation  of 
skills  which  detailed  the  assessment  domain  of  English-to-ASL 
interpretation.  The  domain  was  defined  and  characterized  by 
including  both  ASL  and  interpreting  skills  because  these  two 
competencies  were  interrelated. 

Following  the  task  analysis,  skills  were  clustered  into  related 
groupings.  Eight  groupings  or  Major  Features  of  interpretation  were 
identified:  1)  Numbers,  2)  Fingerspelling,  3)  Classifiers/SASSes,  4) 
Vocabulary,  5)  Structuring  Space,  6)  Grammar,  7)  Interpreting,  and 
8)  Composure  and  Appearance.  A  ninth  category,  Rater's  Judgements, 
was  added  to  provide  the  rater  with  an  opportunity  to  make 
judgements  about  the  overall  performance.  The  eight  Major 
Features,  'Judgements',  and  the  skills  within  each  feature  were 
sequenced  for  ease  of  use  by  the  rater. 

To  further  define  the  domain,  an  error  analysis  was  conducted. 
The  analysis  yielded  a  set  of  relevant  and  representative  errors  that 
might  be  made  by  an  interpreter  for  each  particular  skill.  By  so 
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doing,  the  particular  skill,  that  is,  the  focal  construct,  was  defined 
through  its  list  of  possible  errors. 

The  result  of  completing  this  phase  of  the  research  was  a  first 
draft  diagnostic  assessment  instrument  for  English-to-ASL 
interpretations  which  was  then  reviewed  by  a  panel  of  experts. 

Step  5:  Panel  Review 

A  national  panel  of  14  experts  from  Canada,  8  hearing  and  6 
Deaf,  reviewed  the  first  draft  of  the  instrument.  They  provided  a 
line-by-line  review  of  the  instrument  which  validated  the  list  of 
Major  Features,  the  lists  of  skills  within  each  Major  Feature,  the 
lists  of  possible  errors  associated  with  each  skill,  the  sequencing  of 
the  skills,  and  the  clarity  of  the  writing. 

Following  the  panel  review,  the  panel  members  indicated  that 
the  instrument  filled  a  necessary  gap  in  assessment  methods  not 
currently  available  for  English-to-ASL  interpretation.  As  well,  they 
stated  the  instrument  was  comprehensive  and  had  a  wide  range  of 
possible  uses. 

Based  on  the  experts'  written  and  oral  comments  the 
instrument  was  revised.  Definitions  were  added  to  assist  in 
defining  each  skill  more  clearly,  thereby  assisting  raters  in  the 
rating  process. 


Step  6:  Field  Tests 

Two  field  tests  were  conducted.  Sample  English-to-ASL 
interpretations  (one  novice  and  one  expert)  were  used  in  Field  Test  1 
and  Field  Test  2.  The  same  novice  was  used  in  both  field  tests.  The 
expert  in  Field  Test  1  was  non-certified  and  in  Field  Test  2  was 
replaced  with  a  certified  interpreter.  These  interpreters  provided 
an  interpretation  on  video  tape  of  a  predetermined  text 
encompassing  a  sample  of  skills  from  each  Major  Feature.  Using  the 
second  and  third  drafts  of  the  instrument,  these  samples  were  then 
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assessed  by  expert  hearing  raters  for  Field  Test  1  and  Field  Test  2, 
respectively. 

Field  Test  1  included  two  hearing  raters  from  Alberta,  Canada. 
Field  Test  2  included  seven  hearing  raters  from  California,  United 
States.  The  raters'  responsibilities  were  to  validate  the 
definitions,  further  validate  the  skills  and  errors  within  each  Major 
Feature,  and  for  the  first  time  actually  assess  interpretation 
samples  using  the  instrument.  The  raters  in  Field  Test  2  also 
conducted  a  line-by-line  review  of  the  rater's  instruction  manual 
which  was  developed  after  Field  Test  1  to  explain  the  instrument 
and  the  rating  process  to  potential  raters. 

The  raters  conducted  their  assessments  individually,  allowing 
the  researcher  to  observe,  most  of  them,  while  they  assessed  the 
video  taped  interpretations  and  to  discuss  the  process  upon 
completion  of  their  assessments.  These  discussions  and 
observations  provided  feedback  about  the  instrument  and  its  utility 
in  addition  to  the  raters  written  comments. 

Based  on  the  different  ways  the  raters  used  the  rating  scale 
and  following  further  consultation  with  them,  the  rating  scale  was 
revised  after  each  field  test.  Following  discussions  with  the  raters 
used  in  Field  Test  1,  a  two-tier  rating  scale  was  developed.  The 
first  was  at  the  skill  level  with  points  on  the  scale  to  indicate 
whether  the  skill  was  demonstrated,  and  if  it  was  demonstrated, 
whether  it  was  accurately  or  inaccurately  executed.  At  the  error 
level  a  4-point  scale  was  developed  which  included  both  the 
frequency  and  severity  of  errors  within  the  same  scale.  Following 
Field  Test  2,  the  rating  scale  was  kept  the  same  but  the  definitions 
were  refined  to  clarify  the  points  on  the  rating  scale. 

Based  on  the  written  and  spoken  comments  of  the  raters,  the 
instrument  was  revised  after  Field  Test  1  and  again  after  Field  Test 
2,  resulting  in  the  fourth  draft  of  the  instrument  and  the  second 
draft  of  the  rater's  instruction  manual. 
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Limitations  of  the  Study 

The  purpose  of  this  study  was  to  develop  a  diagnostic 
instrument  to  assess  English-to-ASL  interpretations.  The  small 
number  of  experts  used  in  the  development  of  the  instrument,  the 
panel  review  (N=14)  and  the  field  studies  (N=2  and  N=9),  is  one  of 
the  shortcomings  of  this  research.  With  such  small  numbers,  the 
generalizability  of  the  study  is  limited. 

Another  limitation  of  the  research  was  using  only  a  single 
unrehearsed  interpretation  sample  of  a  five-minute  lecture.  This 
type  of  interpretation  sample  is  limited  in  terms  of  its 
generalizability  to  the  overall  competencies  of  interpreters.  It  is 
only  one  sample  of  one  piece  of  work  and  the  sample  duration  is 
short.  Although  the  first  five  minutes  of  any  interpretation  provides 
for  a  fair  assessment  of  the  interpretation  for  that  five  minutes,  it 
may  not  reflect  a  fair  assessment  of  interpreters'  overall  abilities. 
That  is  to  say,  that  during  many  interpreting  experiences,  in  the 
first  five  minutes  of  work  interpreters  are  not  yet  familiar  with  the 
content  of  the  presentation  or  the  style  of  the  speaker.  In  longer 
and/or  rehearsed  interpretation  samples,  the  interpreter  may 
perform  differently. 

The  instrument  was  designed  for  hearing  raters  to  rate  the 
equivalency  of  the  ASL  target  message  to  the  English  source 
message.  This  limits  the  usability  of  the  instrument  because  the 
number  of  hearing  raters  with  the  required  expertise  is  small.  Using 
Deaf  raters  could  increase  the  pool  of  potential  raters;  however, 
they  do  not  have  access  to  the  English  message  in  the  same  way  as 
hearing  raters  and,  therefore,  can  not  rate  in  the  same  way.  Only 
hearing  raters  can  use  the  instrument  in  its  present  form. 

Step  7:  Findings  and  Conclusions 

Based  on  the  findings,  preliminary  indications  suggest  that  the 
instrument  is  usable  and  that  it  provides  a  standardized  approach  to 
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assessing  interpretations  with  differing  levels  of  skills  from 
English  to  ASL.  According  to  the  raters,  the  instrument  provides  the 
structure  and  detail  to  look  for  a  wide  variety  of  skills  and  errors. 

In  addition,  it  provides  a  method  with  which  to  offer  feedback  to 
interpreters  and  can  provide  consistency  in  the  interpretation  field 
as  to  the  criteria  for  successful  English-to-ASL  interpretations. 

Preliminary  indications  suggest  that  at  the  macro-level,  the 
skill  level,  there  is  low  consistency  among  raters  on  the  rating  of 
the  skills.  This  finding  is  consistent  with  Culton's  (1981)  research 
on  the  RID  exam  when  it  was  a  diagnostic  examination  and  had  a  list 
of  skills  rated  on  a  five-point  scale.  However,  at  the  micro-level  or 
error  level,  by  further  defining  each  skill  with  its  relevant  and 
representative  possible  errors,  the  rating  consistency  among  raters 
significantly  increased  in  Field  Test  2  from  64%  at  the  skill  level  to 
77%  at  the  error  level  for  interpretation  2  and  from  54%  to  95%  for 
interpretation  3. 

With  respect  to  the  English  speech  presented  on  the  five- 
minute  audio  tape  used  in  the  validation  process,  the  raters 
indicated  that  the  speaker  was  well-paced  and  that  the  length  and 
content  of  the  speech  was  conducive  to  viewing  samples  of 
interpretation  skills  in  each  of  the  eight  Major  Features.  Thus,  the 
stimulus  used  for  the  assessments  was  appropriate,  with  no  need  to 
change  the  stimulus  tape. 

The  time  required  to  complete  the  assessments  ranged  from  1 
hour  and  10  minutes  to  3  hours  and  5  minutes,  with  an  average  of 
about  2  hours.  Thus,  the  assessments  were  accomplished  in  a 
reasonable  amount  of  time  indicating  the  instrument  is  practical 
when  assessing  a  5-minute  interpretation  sample. 

Implications  for  Practice 

The  implications  for  practice  are  two-fold.  First,  the  lists  of 
identified  skills,  described  by  their  associated  possible  errors  and 
definitions,  have  been  agreed  upon  by  Deaf  and  hearing  experts. 
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These  lists  of  skills  can  be  used  by  interpreter  educators  in  several 
ways.  For  example,  educators  can  compare  their  current  curriculum 
and  the  skills  they  currently  teach  to  those  included  in  this 
diagnostic  assessment  instrument.  In  addition,  they  can  use  the 
identified  skills  to  guide  curriculum  development  within 
interpreting  programs  with  the  possibility  of  providing  standardized 
skill  development  across  programs  in  North  America. 

The  second  implication  for  practice  is  that  at  this  point  the 
diagnostic  instrument  is  ready  for  its  final  phase  of  development. 

As  will  be  seen  in  the  next  section,  the  instrument  must  be  used  in 
one  more  trial  before  it  is  fully  operational.  However,  every 
indication  thus  far  demonstrates  the  instrument  is  usable  and 
practical.  However,  before  this  statement  can  be  made  with 
complete  confidence,  the  final  phase  of  development  must  be 
completed.  Once  this  has  been  done,  then  the  implications  for 
practice  can  be  developed  more  completely. 

Implications  for  Research 
Required  Research 

The  fourth  draft  of  the  instrument  is  ready  for  a 
comprehensive  field  study  to  further  validate  its  utility.  One  of  the 
next  steps  in  the  continuation  of  this  research  is  to  use  raters  with 
varying  degrees  of  experience.  The  field  studies  conducted  thus  far 
used  raters  with  a  minimum  of  five  years  of  interpreter  education 
experience.  The  potential  pool  of  raters  will  increase  if  raters  with 
less  than  five  years  of  experience  are  used.  Also  a  larger  number  of 
sample  interpretations  needs  to  be  used.  By  increasing  the  number 
of  samples  assessed  by  a  larger  number  of  raters,  the 
generalizability  of  the  results  will  be  enhanced  and  the  instrument 
will  ultimately  be  improved  based  on  the  revisions  resulting  from 
the  collection  of  this  data. 
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Along  with  further  validation  of  the  instrument  itself,  there  is 
a  need  to  ask  interpreters  undergoing  the  assessment  for  their  input 
to  be  certain  it  addresses  their  needs.  How  should  the  assessment 
results  be  shared  with  them?  For  example,  should  it  be  done  in 
person  or  in  writing?  Can  it  be  done  by  mail?  What  information  can 
raters  provide  to  interpreters  to  assist  in  their  skill  development? 

Improving  the  Instrument 
Knowledge-Rich  Major  Features 

The  Major  Features  of  Grammar  and  Interpreting  are  more 
knowledge-rich  and  have  lower  agreement  than  more  knowledge-lean 
features  such  as  Fingerspelling  and  Numbers.  Can  the  skills  and 
possible  errors  in  these  Major  Features  be  further  delineated,  and  if 
so,  would  rater  agreement  increase? 

Instrument's  Self-Instructional  Qualities 

The  instrument  with  its  accompanying  manual  is  designed  to 
be  self-instructional  for  the  rater.  Developing  a  video  tape 
displaying  segments  of  actual  interpretations  and  discussing  the 
rating  procedures  would  be  one  way  to  enhance  the  self- 
instructional  nature  of  the  instrument. 

As  the  instrument  is  designed  thus  far,  raters  and/or 
interpreters  are  responsible  for  developing  activities  based  on  the 
assessment  results  that  will  improve  future  performance.  This 
relies  on  the  expertise  of  the  individuals  to  develop  an  adequate  plan 
for  improvement.  Another  way  to  enhance  the  utility  of  the 
instrument  is  to  add  a  component  within  the  instruction  manual  on 
specific  remedies  and  activities  which  address  particular 
weaknesses  and  that  might  improve  interpreting  performance. 
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Time  Required  to  Rate  Interpretations 

More  research  must  be  conducted  to  determine  the  amount  of 
time  required  to  rate  interpretations.  For  example,  how  long  does  it 
take  raters  with  various  experience  levels  to  use  the  instrument  and 
rate  interpretations?  What  is  the  average  length  of  time  required  to 
assess  a  sample  interpretation  after  rating  five  or  ten 
interpretations?  At  what  point  does  a  plateau  occur?  Does  rating 
expert  interpretations  require  less  time  than  rating  novice 
interpretations?  If  so,  what  is  the  impact  of  this  finding  on  the 
number  of  interpretation  samples  raters  can  rate  within  a  given 
period  of  time? 


Increasing  the  Pool  of  Raters 

The  instrument  was  designed  for  hearing  interpreter  educators 
to  compare  the  message  equivalency  of  the  target  message,  ASL, 
with  the  source  message,  English.  ASL  is  the  language  of  the 
majority  of  North  American  Deaf  people  and  much  of  the  instrument 
assesses  ASL  skills,  albeit  in  the  context  of  interpretation.  Can 
Deaf  people  be  used  as  qualified  raters  to  assess  interpretations  and 
increase  the  pool  of  potential  raters?  What  qualifications  are 
required?  Can  they  use  the  instrument  in  part  or  in  whole?  What  do 
they  need  to  conduct  the  assessment?  For  example,  is  it  sufficient 
to  use  an  agreed-upon  expert  interpretation  as  a  comparison  to 
assess  interpretations?  If  Deaf  people  are  chosen  as  raters  to  use 
the  instrument,  how  might  they  be  different  in  their  rating 
behaviors? 


Developing  Similar  Instruments 

Signed  and  spoken  language  interpreters,  using  different 
languages  from  ASL  and  English,  can  develop  similar  instruments 
using  the  methodology  in  this  research.  The  most  obvious  is  a 
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companion  instrument  to  assess  ASL-to-English  interpretations.  In 
addition,  spoken  language  interpreters  could  develop  a  diagnostic 
instrument  to  assess  French-to-English  or  English-to-French 
interpretations;  sign  language  interpreters  can  develop  an 
instrument  to  assess  German  Sign  Language-to-German  or  German- 
to-German  Sign  Language. 

Effect  on  Interpreters  Receiving  Feedback  From  Raters 

The  scope  of  the  study  did  not  include  an  investigation  into  the 
impact  of  undergoing  the  assessment  on  the  interpreter  and  the 
resulting  feedback.  Once  the  assessment  is  completed,  the  rater 
must  provide  feedback  and  direction  to  the  interpreter.  This  was 
never  asked  of  the  raters  involved  in  the  field  tests;  the  research 
stopped  short  of  this.  One  way  to  accomplish  this  is  for  raters  to 
use  the  Cross  Reference  Index  in  the  back  of  the  instrument  and 
examine  the  rating  of  the  skills  appearing  in  more  than  one  Major 
Feature.  By  checking  the  rating  consistency  for  each  of  these  skills, 
the  rater  could  develop  a  plan  of  action  to  assist  interpreters  in 
developing  their  skills  by  pointing  out  to  them  their  strengths  and 
weaknesses  across  Major  Features.  Also,  the  rater  can  refer 
interpreters  to  resources  that  will  assist  them  in  developing  their 
skills  further,  such  as  video  tapes,  articles,  journals,  and 
workshops.  What  information  is  the  most  useful  and  effective  in 
improving  interpreters'  skills? 

Questions  for  future  research  are,  for  example,  once  diagnostic 
information  is  provided  to  interpreters,  what  is  the  effect?  When 
interpreters  are  assessed  a  second  time,  have  they  improved? 


Key  Skills  of  Novices  and  Experts 

Further  research  is  needed  in  the  area  of  specific  skills  which 
experts  demonstrate  and  the  sequence  in  which  they  acquire  these 
skills.  Several  questions  need  to  be  addressed.  For  example,  do 
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particular  skills  consistently  show  up  in  certified  interpreters  that 
are  not  demonstrated  by  non-certified  interpreters?  Are  there  key 
skills  within  each  Major  Feature  or  within  one  or  two  Major  Features 
that  will  identify  expert  interpretations?  Should  some  skills  be 
weighted  more  heavily  than  others  in  terms  of  an  overall  rating? 

Future  research  can  be  conducted  to  determine  the  sequence  of 
development  among  interpreters  with  various  years  of  experience. 
For  example,  do  interpreters  with  less  than  two  years  of  experience 
display  particular  errors  not  exhibited  in  interpreters  with  five  or 
more  years  of  experience?  What  skills  do  more  experienced 
interpreters  display  that  less  experienced  interpreters  do  not?  Can 
the  following  questions  be  answered?  What  skills  do  interpreters 
develop  first?  What  skills  do  interpreters  develop  last?  At  what 
point  do  the  most  significant  changes  in  skill  level  occur?  From  the 
answers  to  these  questions  a  sequence  of  skill  development  could  be 
determined  with  the  possibility  of  enhancing  the  curricula  available 
to  interpreter  training  programs  by  determining  in  what  sequence 
ASL/English  interpreting  skills  should  be  taught. 
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Appendix  B 

Video  Tapes  With  Interpretation  Models 


1  1  7 


Commercially  Produced 

Interpreter  model  series:  English  to  ASL.  (1985).  Burtonsville, 
Maryland:  Sign  Media. 

Performing  arts:  Modelling  tape  3A.  (1989).  Salem,  OR:  Sign 
Enhancers. 

Performing  arts:  Modelling  tape  3B.  (1989).  Salem,  OR:  Sign 
Enhancers. 

Taylor,  M.  M.  (Director).  (1984).  National  festival  of  the  arts 
[Videotape].  Edmonton,  Alberta:  Grant  MacEwan  Community 
College. 

Taylor,  M.  M.  (Director)  &  Chauvet,  V.  (Speaker).  (1986).  American 
Sign  Language:  Grammatical  features  [Videotapes  1-10]. 
Edmonton,  Alberta:  Grant  MacEwan  Community  College. 

Taylor,  M.  M.  (Director).  (1990).  Workshops  for  post-secondary 

interpreters  [Videotapes  1-4].  Edmonton,  Alberta:  Grant  MacEwan 
Community  College. 

Taylor,  M.  M.  (Director)  &  Stratiy,  A.  J.  (Speaker).  (1987-1988). 
American  Sign  Language  for  interpreters  [Videotapes  1-27]. 
Edmonton,  Alberta:  Grant  MacEwan  Community  College. 

Voice  to  sign  interpretation/transliteration: _ Modeling  tape  ,2 A. 

(1989).  Salem,  OR:  Sign  Enhancers. 
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Voice  to  sign  interpretation/transliteration:  Modeling  tape  2B. 

(1989).  Salem,  OR:  Sign  Enhancers. 

Voice  to  sign  interpretation/transliteration:  Modeling  tape  2C. 

(1989).  Salem,  OR:  Sign  Enhancers. 

Televised  Programs  Copied  onto  Video  tape 

ABC-Retinitis  Pigmentosa  Telethon.  (1983).  California. 

CBC-Air  Canada  Sports  Award.  (1987).  Canada. 

CBC-Alberta  Newsweek.  (1988  -  1989).  Edmonton. 

CBC-Calgary  News.  (1987,  March).  Alberta. 

CBC-Edmonton  Tornado  News.  (1987,  July).  Alberta. 

CTV-Winnipeg  News.  (1987,  May).  Manitoba. 

Deaf  Family  Magazines.  (1987).  United  States. 

Interpreted  Religious  Program  with  Jerry  Falwell.  (1984, 
September).  California. 


QCTV-ln  Touch  With  Deaf-Blindness.  (1989,  June  24).  Alberta. 
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Appendix  C 

Panel  Members  Qualifications  Forms 

HEARING  EXPERT  QUALIFICATIONS 


Name: 

Address: 

City  &  Province: 
Postal  Code: 
Phone  (H): 
Phone  (W): 
DISC: 

Birthdate: 


Is  ASL  the  language  you  learned  first?  Yes _  No 

Is  English  the  language  you  learned  first?  Yes _  No 

Do  you  have  experience  teaching  interpreters?  Yes _  No 

If  yes,  then 

What  year  did  you  start  teaching  interpreters?  _ 

How  many  years  have  you  taught  interpreters?  _ 

Do  you  have  experience  teaching  in  an  interpreter  program? 
Yes _  No _ 

Do  you  have  experience  teaching  employed  interpreters? 

Yes _  No _ 


What  local  interpreting  organizations  do  you  belong  to? 


What  national  interpreting  organizations  do  you  belong  to? 


What  national  interpreter  educator  organizations  do  you  belong  to? 


Do  you  hold  a  currently  valid  interpreter's  certification? 
Yes _  No _ 

If  yes,  what  certification(s)?  _ 

Are  you  currently  interpreting?  Yes  _  No _ 
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DEAF  EXPERT  QUALIFICATIONS 


Name: 

Address: 

City  &  Province: 
Postal  Code: 
Phone  (H): 
Phone  (W): 
DISC: 

Birthdate: 


Is  ASL  your  first  language?  Yes _  No 

Is  English  your  first  language?  Yes _  No 

Do  you  have  experience  teaching  ASL?  Yes _  No 

If  yes,  then 

What  year  did  you  start  teaching  ASL?  _ 

How  many  years  have  you  taught  ASL?  _ 

To  whom  have  you  taught  ASL? 


Adults 

Yes 

No 

Children 

Yes 

No 

Interpreters 

Yes 

No 

What  local  deaf  organizations  are  you  a  member  of? 


What  national  deaf  organizations  are  you  a  member  of? 


How  many  years  have  you  used  interpreters?  _ 

Have  you  had  experience  using  new  interpreters?  Yes _  No 

Have  you  had  experience  using  experienced  interpreters?  Yes _  No _ 
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Appendix  D 

Forms  Used  to  Provide  Consistency  in  Responses  From  Experts 


MAJOR  FEATURES  RATING  FORM 


I  agree  that  all  the  Major  Features  needed  to  assess  interpreters  are 

incorporated  in  this  assessment  instrument.  Yes _ No 

If  no,  then,  I  would  add  the  following  Major  Features: 


I  agree  that  all  the  Major  Features  listed  are  necessary. 

Yes _  No 

If  no,  then,  I  would  recommend  deletion  of  the  following  Major 
Features: 


Please  feel  free  to  use  the  back  of  this  paper  to  make  additional 
comments  which  may  improve  this  instrument. 
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Page  1 


EXPERT  RATING  FORM 


Expert  name: 

Checklist  evaluating:  _  Pages: 

Comprehensiveness  of  skills 

I  agree  that  this  checklist  of  skills  is  comprehensive. 

Yes _  No _ 

If  no,  then  I  would  add  the  following  skills: 


Comprehensiveness  of  skills 

I  agree  that  all  the  skills  should  be  kept  in  this  list. 
Yes _  No _ 

If  no,  then  I  would  delete  the  following  skills: 
Number  Skill 
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Expert  name: 


Page  2 


Checklist  evaluating:  _  Pages: _ 

Comprehensiveness  of  errors 

I  agree  that  the  errors  in  this  checklist  are  comprehensive. 

Yes _  No _ 

If  no,  then  I  would  add  the  following  errors: 

Number  Error 


Comprehensiveness  of  Errors 

I  agree  all  the  errors  in  this  checklist  should  be  kept  on  the  list. 
Yes _  No _ 

If  no,  then  I  would  delete  the  following  errors: 

Number  Error 
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Page  3 


Sequencing 

I  agree  with  the  sequencing  of  this  checklist. 

Yes _  No _ 

If  yes,  then, 

Essential  skills  are  numbers 


Desirable  skills  are  numbers 


If  no,  then,  my  sequence  is: 
Essential  skills  are  numbers 


Desirable  skills  are  numbers 


Clarity  of  writing 

I  agree  that  the  skills  and  errors  in  this  checklist  are  clearly  stated. 
Yes _  No _ 

If  no,  then  I  would  recommend  the  following  changes. 

Element  #  Recommended  change 
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Appendix  E 

Audio  Tape  Transcript 

"Spreading  Love  Around  the  World" 
by  Leanne  Walls 

(Adapted  from  Looking  Out.  Looking  In 
by  Ronald  B.  Adler  and  Neil  Towne,  1990,  p.  47"*) 

I  had  finally  made  it  to  class.  1989  had  to  have  the  coldest 
January  on  record.  It  was  about  -28  and  there  were  dead  cars  all 
along  my  route  to  school.  The  city  looked  kind  of  futuristic  with 
steam,  exhaust  and  ice  fog  hanging  in  the  air.  Anyway,  I  hadn't 
wanted  to  budge  out  of  home  and  I  was  cranky  and  thinking  that 
'today's  lecture  better  be  good!" 

The  prof  started  class  with  a  story  called  "Love  and  the 
Cabbie"  by  Art  Buchwald.  It  went  something  like  this: 

Art  was  going  to  New  York  to  visit  a  friend.  The  friend  met 
him  at  JFK  airport  and  they  got  into  a  beaten  up  old  cab.  When  they 
got  to  their  destination,  the  friend  handed  the  cabbie  $15.00  and 
said,  "Thank  you  for  the  ride.  You  did  a  superb  job  of  driving!"  The 
cabbie  was  stunned  for  a  second  and  then  said,  "Are  you  a  wise  guy 
or  something?" 

Art's  friend  replied,  "No,  I  truly  admire  the  way  you  stay  cool 
in  traffic." 

"Yeh,"  said  the  cabbie  as  he  pulled  away. 

Art  asked  "What  was  that  about?" 

Well,  his  friend  replied,  "I'm  trying  to  bring  love  back  to  new 
York.  I  believe  it's  the  only  thing  that  can  save  the  city." 

"How  can  one  person  save  New  York?" 


1  Looking  Out,  Looking  In  (1990).  6th  edition,  by  Ronald  B.  Adler,  and 
Neil  Towne.  Holt,  Rinehart  and  Winston,  Inc.  Orlanda,  FLA  32887.  p.47 
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"It's  not  one  person.  I  believe  I  made  that  taxi  driver's  day. 
Now  suppose  he  has  twenty  fares.  He's  going  to  be  nice  to  those 
twenty  people  because  someone  was  nice  to  him.  Those  fares  in  turn 
will  be  kinder  to  their  employees  or  shopkeepers  or  waiters  or  even 
their  own  families.  Eventually,  the  goodwill  could  spread  to  1,000 
people.  Not  bad,  hey?" 

Art  said,  "But  you're  depending  on  that  taxi  driver  to  pass 
your  goodwill  to  others." 

His  friend  said,  "I'm  not,  I'm  not  depending  on  it.  I'm  aware, 
that  the  system  is  not  foolproof,  so  I  might  deal  with  10  different 
people  today.  If,  out  of  10,  I  can  make  three  happy,  then  eventually  I 
can  indirectly  influence  the  attitudes  of  3,000  more." 

"Hmm,  sounds  good  on  paper,"  Art  admitted,  "but  I'm  not  sure 
it  works  in  practice." 

His  friend  figured  nothing  was  lost  if  if  it  didn't  work.  And, 
ah,  Art  was  saying  to  him  then  that  he  was  sort  of  some  kind  of  nut. 

And  his  friend  said,  "Well,  that  shows  how  cynical  you've 
become,  Art.  I've  made  a  study  of  this.  The  thing  that  seems  to  be 
lacking,  besides  money  of  course,  for  our  postal  employees,  for 
example,  is  that  no  one  tells  people,  who  work  for  the  post  office, 
what  a  good  job  they're  doing." 

"But  they're  not  doing  a  good  job." 

"They're  not  doing  a  good  job  because  they  feel  no  one  cares 
if  they  do  or  not.  Why  shouldn't  someone  say  a  kind  word  to  them?" 

They  were  walking  past  a  big  building  in  the  process  of  being 
built  and  passed  five  workmen  eating  their  lunch.  Art's  friend 
stopped.  "That's  a  magnificent  job  you  men  have  done.  It  must  be 
difficult  and  dangerous  work." 

The  five  men  sort  of  looked  at  him  suspiciously. 

"When  will  it  be  finished?" 

"June,"  a  man  grunted. 

"Ah.  That  really  is  impressive.  You  must  all  be  very  proud." 
And  on  they  went. 
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Art  said  to  him,  "I  haven't  seen  anyone  like  you  since  the 
movie  The  Man  from  La  Mancha.'" 

And  then  Art's  friend  was  saying  to  him  that  when  the  men 
figure  out  what  he'd  said  they're  gonna  feel  better  and  somehow  the 
this  whole  city  would  benefit  from  from  his  kind  words. 

Art  said,  "But  you  can't  do  this  all  alone!  You're  just  one 

man." 

Well,  his  friend  replied,  "The  most  important  thing  is  not  to 
get  discouraged.  Making  people  in  the  city  become  kind  again  is  not 
an  easy  job,  but  if  I  can  enlist  other  people  in  my  campaign..." 

Art  said,  "You  just  winked  at  a  very  plain  looking  woman." 

"Yes,  I  know,"  his  friend  replied.  "And  if  she's  a 
schoolteacher,  her  class  will  be  in  for  a  fantastic  day." 

Anyway,  that  story  has  always  stuck  with  me  and  and  it 
made  venture  made  venturing  out  that  day  very  worthwhile. 
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TAYLOR  S  DIAGNOSTIC 

INSTRUMENT: 


English  to 

American  Sign  Language 
Interpretation  Assessment 

by  Marty  M.  Taylor 


Date  interpretation  was  video  taped: 

Video  tape  number  or  name: _ 

Do  you  know  this  interpreter?  _ 

If  yes,  in  what  capacity?  _ 

Rater  name: _ 

Rating  date:  _ _ 

Time  started:  _ _ 

Numbers  of  breaks: _ 

Length  of  breaks:  _ 

Time  finished:  _ _ _ 


For  research  purposes  only 


June  1993 


June  1993 
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FINGERSPELLING 


Rating  Key 


NA  =  Not  applicable,  not  in  the  target  language,  ASL 

Y  =  Yes,  the  skill  was  performed  accurately  100%  of 

the  time 

N  =  No,  the  skill  was  not  performed  accurately  100% 

of  the  time 


m 

O 

1 

2 

3 

4 


Not  applicable,  not  in  the  target  language,  ASL 


No  errors  observed 
Low  frequency, 
High  frequency, 
Low  frequency, 
High  frequency  , 


low  severity 
low  severity 
high  severity 
high  severity 


Do  not  copy 


For  research  purposes  only  © 


M.  Taylor 


June  1993 
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FINGERSPELLING 


1.  DEF:  If  the  word  is  misspelled  or  if  there  are  missing  or  added  letters,  the  word  is 
spelled  inaccurately.  Also,  if  the  wrong  word  is  spelled  such  as  C-A-B-B-A-G-E  for  the 
English  word  'cottage',  it  is  an  error. 


2.  DEF:  Acronyms  are  signed  in  several  different  ways.  A  slight  pause  between  letters 
with  a  slight  circular  movement  indicates  the  signing  of  initials.  If  acronyms  are 
commonly  used,  they  are  spelled  as  if  they  are  words  instead  of  individual  letters  (e.g. 
C-S-U-N  for  California  State  University,  Northridge)  assuming  the  addressee  is  equally 
familiar  with  the  acronym  as  the  interpreter.  When  the  speaker  says  the  entire  name 
such  as  Grant  MacEwan  Community  College  and  the  addressee  is  familiar  with  this 
institution,  then  the  formation  of  G-M-C-C  would  be  made  with  a  pause  between  letters 
and  small  circular  motions  with  each  letter  indicating  the  speaker  mentioned  the  words 
of  the  college,  not  just  the  initials.  Another  example  that  might  be  initialized  locally  but 
not  nationally,  is  the  local  service  agency  providing  interpreters  for  deaf  and  hearing 
people. 

3.  DEF:  Omissions  are  considered  errors.  Words  that  should  be  spelled  should  not  be 
deleted.  These  omissions  can  occur  for  a  variety  of  reasons  such  as  when  the  interpreter 
is  not  familiar  with  the  information  (e.g.  names)  or  the  information  is  too  fast  to  keep 
up.  For  example,  assuming  name  signs  are  not  already  established,  if  people  are 
introducing  themselves,  their  names  should  be  spelled.  All  of  the  information  present  in 
the  source  message  should  be  in  the  target  message.  There  should  be  no  deletions. 


Do  not  copy 


For  research  purposes  only  © 


M.  Taylor 


June  1993 
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FINGERSPELLING 


1 .  Words  are  fingerspelled  accurately  . NA  Y  N 

Error 

1.A  Words  are  fingerspelled  inaccurately  (e.g.  misspelled 
words,  letters  are  missing  such  as  P-I-N...P-L-E  for 
pineapple  . NA  0  12  3  4 


2.  Acronyms  are  formed  accurately  (e.g.  G.U.A.A.,  A.V.L.I.C.) . NA  Y  N 

Error 

2. A  Acronyms  are  formed  inaccurately  . NA  0  12  3  4 


3.  Words  that  should  be  spelled  are  present  . NA  Y  N 

Error 

3. A  Words  are  deleted  . NA  0  12  3  4 


Do  not  copy 


For  research  purposes  only  © 


M.  Taylor 


June  1993 
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4.  DEF:  Proper  names  (e.g.  titles  of  films  and  books)  should  be  spelled  unless 
previously  agreed-upon  signs  exist.  Proper  nouns  include  regional  variations; 
therefore,  what  is  an  accurate  sign  in  one  location  may  not  be  accurate  in  another.  For 
example,  if  interpreting  in  Canada,  the  city  of  Winnipeg  is  most  often  signed,  but  if 
interpreting  in  the  United  States,  Winnipeg  would  probably  be  spelled  assuming  the 
addressee  is  not  familiar  with  the  signs  for  Canadian  cities  such  as  Winnipeg.  At  times 
speed  of  the  speaker  becomes  a  problem  and  is  evident  in  the  interpretation  when  words 
that  should  be  spelled  are  signed  instead,  hoping  to  save  time  in  the  interpretation. 

When  this  occurs  it  is  an  error.  Another  interpretation  phenomenon  is  splitting  an 
English  word,  that  is,  for  example,  spelling  half  of  the  word  and  signing  the  other  half  as 
in  'happiness'  when  it  is  signed  HAPPY  +  N-E-S-S.  Again  this  is  not  accurate.  The 
entire  word  should  either  be  signed  or  spelled  but  not  a  mixture  of  both. 


5.  DEF:  Fingerspelling  can  be  used  for  both  importance  and  emphasis  (e.g.  W-O-N-T), 
however  it  should  not  be  overused,  meaning  it  is  used  so  often  as  to  lose  its  impact, 
unless  of  course  this  is  the  intent  of  the  speaker.  The  spelling  should  be  noticeably 
different  from  other  parts  of  the  interpretation  often  with  strong  precise  movements 
from  one  letter  to  the  next  with  the  eye  gaze  directed  either  toward  the  addressee  or  the 
hand. 


Do  not  copy 


For  research  purposes  only  © 


M.  Taylor 
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4.  Accurately  chooses  to  fingerspell  those  words  which  should  be 

spelled  and  signs  those  concepts  which  should  be  signed  . NA  Y  N 

Errors 

4. A  Spells  words  that  should  be  signed  (e.g.  relies  excessively 

on  spelling)  . NA  0  12  3  4 

4.B  Signs  words  that  should  be  spelled  (e.g.  signs  instead  of 

spells  C-A-N-A-D-l-A-N  pause  A-l-R-L-l-N-E-S)  . NA  0  12  3  4 


5.  Fingerspelling  is  used  for  emphasis  accurately  . NA  Y  N 

Errors 

5. A  Eye  gaze  is  inaccurate  . NA  0  12  3  4 

5.B  Rhythm  or  cadence  is  inaccurate  . NA  0  12  3  4 

5.C  Fingerspelling  with  emphasis  is  overused  . NA  0  12  3  4 
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6.  DEF:  Letters  should  be  formed  with  an  even  pace  or  pause  between  each  letter, 
including  double  letters,  such  as  B-B  and  O-O.  Pauses  between  the  words  are  longer  in 
duration  than  those  between  letters.  If  the  words  are  spelled  without  pauses  or  the 
pauses  appear  at  uneven  intervals,  the  rhythm  is  inaccurate. 


7.  DEF:  If  the  spelling  is  clear,  the  movement  is  complete  and  each  individual  letter  is 
formed.  For  example,  in  the  fingerspelled  word  L-A-M-B  there  should  be  no  extra 
letters  present  as  in  L-A-C-M-B.  The  individual  letters  are  unclear,  if  it  is  difficult  to 
comprehend  the  spelling. 
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6.  The  spelling  rhythm  is  uniform  within  and  between  words  . NA  Y  N 

Errors 

6. A  The  pauses  between  words  are  not  present  (e.g.  L-l-T-T- 

L-E-H-O-U-S-E-O-N-T-H-E-P-R-A-l-R-l-E)  . NA  0  12  3  4 

6.B  The  pauses  between  words  are  uneven  (e.g.  L-l-T-T-L-E- 
pause  pause  pause  H-O-U-S-E-O-N-T-H-E  pause  -P-R- 

A-l-R-l-E)  . NA  0  12  3  4 

6.C  The  pauses  within  words  are  uneven  (e.g.  F-R-A-N-pause 

K-L-l-N) . NA  0  12  3  4 


7.  Individual  letters  are  clearly  presented  . NA  Y  N 

Error 

7. A  The  individual  letters  are  unclear  (e.g.  sloppy,  tense, 

cramped)  . NA  0  12  3  4 


Do  not  copy 


For  research  purposes  only  © 


M.  Taylor 


June  1993 


137 


8.  DEF:  The  hand  should  be  held  in  a  visually  comfortable  position  with  the  arm  bent 
and  the  hand  slightly  in  front  of  the  shoulder  area,  not  the  face.  The  arm  should  not  be 
regularly  extended  away  from  the  body  toward  the  addressee.  When  fingerspelling,  the 
hand  should  either  remain  stationary  or  move  slightly  outward  from  the  shoulder  area  of 
the  body  for  both  left  and  right  interpreters.  It  is  inaccurate  to  regularly  fingerspell 
outward  to  inward.  The  arm  should  not  move  upward  (e.g.  spelling  above  the  shoulder) 
or  downward  (e.g.  down  near  the  waist).  The  movement  of  the  hand  should  not  jump  or 
twist.  The  palm  of  the  hand  or  the  profile  of  the  hand  (especially  to  prevent  repetitive 
motion  injuries),  not  the  back  of  the  hand,  should  be  directed  toward  the  addressee. 


9.  Def:  The  speed  of  spelling  is  highly  dependent  on  the  addressee's  skill  and/or 
preference.  If  the  deaf  consumer  is  a  native  or  near  native  signer  the  speed  of  the 
spelling  is  less  important.  Slow  fingerspelling  can  annoy  the  consumer  or  appear 
patronizing.  Slow  fingerspelling  may  be  used  accurately  when  importance  or  emphasis 
is  shown.  In  this  case  it  is  accurate  to  fingerspell  slowly.  Fast  fingerspelling  is 
acceptable  if  it  is  readable.  If  novel  information  is  conveyed  using  the  same 
fingerspelling  rate  as  words  that  are  familiar  and  commonly  used,  this  would  be 
considered  too  fast. 


Do  not  copy 


For  research  purposes  only  © 


M.  Taylor 


June  1993 


138 


8.  The  hand  is  stationary  and  in  an  appropriate  location  . NA  Y  N 

Errors 

8. A  Excessive  vertical  movements  are  present  . NA  0  12  3  4 

8.B  Excessive  horizontal  movements  are  present  . NA  0  12  3  4 

8.C  Extraneous  movements  are  present  (e.g.  bounces,  twists 

wrist  excessively)  . NA  0  12  3  4 

8.D  Palm  orientation  is  directed  toward  the  interpreter  . NA  0  12  3  4 

8.E  The  location  of  the  spelling  is  intrusive  (e.g.  hand  in  front 

of  face  or  down  at  waist)  . NA  0  12  3  4 


9.  Words  are  spelled  at  an  understandable  speed  . NA  Y  N 

Errors 

9. A  Words  are  spelled  too  fast  . NA  0  12  3  4 

9.B  Words  are  spelled  distractingly  slow  . NA  0  12  3  4 
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10.  Def:  The  presence  or  absence  of  mouthing  is  a  personal  preference  of  deaf 
consumers.  It  may  or  may  not  be  an  error.  Because  fingerspelling  is  a  way  to  borrow 
from  English,  it  is  permissible  to  mouth  the  words  spelled.  However,  if  individual 
letters  instead  of  words  are  mouthed  or  excessive  mouthing  is  used  (e.g.  unnatural 
mouthing),  it  is  an  error.  If  mouthing  is  present,  the  entire  word,  not  individual 
letters,  should  be  mouthed  and  mouthed  naturally.  Mouthing  should  also  coincide  with 
the  spelling  of  the  word  and  occur  simultaneously  if  it  is  done.  In  other  words  the 
mouthing  should  not  occur  before  or  after  the  spelling. 
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10.  Mouthing  of  fingerspelled  words  is  appropriate  . NA  Y  N 

Errors 

10.A  Mouthing  is  not  present  when  it  should  be  (e.g.  mouth  is 

closed)  . NA  0  12  3  4 

10.B  Mouthing  of  individuals  letters  instead  of  words  . NA  0  12  3  4 

10.C  Excessive  mouthing  is  present  (e.g.  distracting  tongue 

thrusts)  . NA  0  12  3  4 

10. D  Mouthing  is  done  at  a  different  time  from  the  spelling  (e.g. 

before  or  after  the  words  are  spelled)  . NA  0  12  3  4 
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REMINDERS 


-Be  sure  to  refer  to  the  work  or  the  interpretation,  not  the 

interpreter,  when  writing  comments.  Avoid  using  ’yoLi',  for 
example,  "you  signed..."  or  "your  interpretation..."  Instead  use 
"the  interpretation  demonstrated  signs  that..."  or  the 
"interpretation  was..." 

-Be  sure  to  write  both  positive  and  negative  comments  when 
providing  feedback  about  the  interpretation. 

-View  the  tape  at  least  twice  without  sound. 

-Be  sure  to  rate  every  skill  and  every  error. 

-If  the  skill  is  not  present,  then  it  is  marked  NA,  not  applicable,  and 
all  of  the  errors  related  to  this  skill  are  also  marked  NA. 

-Remember  to  rate  the  errors  associated  with  the  skill  it  defines 

only.  The  errors  should  not  be  rated  as  if  they  were  associated 
with  other  skills  For  example,  the  error  5.B  "rhythm  or 
cadence  is  inaccurate"  should  be  rated  in  association  with 
skill  5.  "Fingerspelling  is  used  for  emphasis  accurately".  It 
should  not  be  rated  for  overall  rhythm  or  cadence  of  the 
fingerspelling  in  the  interpretation. 


Do  not  copy 


For  research  purposes  only  © 


M.  Taylor 


June  1993 


142 


NUMBERS 


Rating  Key 


NA  =  Not  applicable,  not  in  the  target  language,  ASL 

Y  =  Yes,  the  skill  was  performed  accurately  100%  of 

the  time 

N  =  No,  the  skill  was  not  performed  accurately  100% 

of  the  time 


ISA 

O 

1 

2 

3 

4 


Not  applicable,  not  in 
No  errors  observed 
Low  frequency, 

High  frequency, 

Low  frequency, 

High  frequency  , 


the  target  language,  ASL 

low  severity 
low  severity 
high  severity 
high  severity 
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NUMBERS 

11.  DEF:  Numbers  are  precise  elements  of  information.  There  is  often  a  lack  of  context 
in  which  to  remember  the  information,  therefore,  often  numbers  can  be  either  incorrect 
or  deleted.  Skill  #11  addresses  the  number  only.  Is  the  English  32  and  the  ASL 
TWENTY-THREE?  If  yes,  then  it  is  inaccurate.  If  the  numbers  are  deleted  then  11. A 
should  be  rated  'NA',  not  applicable. 


12.  DEF:  ASL  has  a  variety  of  numbering  systems.  For  example,  cardinal  (e.g.  7  balls) 
and  ordinal  numbers  (e.g.  first  in  line)  are  signed  differently  from  one  another,  as  are 
numbers  associated  with  height  and  scores  in  sports.  In  order  for  the  interpretation  of 
the  numbering  system  to  be  accurate,  the  palm  orientation  must  be  accurate.  For 
example,  the  signing  of  a  telephone  number,  555-2547,  should  all  be  signed  with  the 
palm  toward  the  addressee.  Twisting  from  palm  out  to  palm  in  should  not  occur.  The 
movement  should  be  accurate  for  each  number  and  numbering  system.  If  talking  about 
the  5-3  baseball  score,  the  movement,  assuming  the  speaker  is  not  supporting  either 
team  should  be  from  the  passive  to  the  dominant  side  and  in  this  case  the  palm  should  be 
facing  the  interpreter.  Finally,  the  location  of  the  number  must  be  accurate.  If  the 
number  discussed  is  player  number  11,  often  it  is  signed  on  the  chest  or  the  upper  arm 
of  the  passive  side,  and  not  signed  in  neutral  space. 
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NUMBERS 

1 1 .  Numbers  are  signed  accurately  . NA  Y  N 

Errors 

11. A  Numbers  are  signed  inaccurately  (e.g.  17  instead  of  16)  . NA  0  12  3  4 

11.B  Numbers  are  deleted  . NA  0  12  3  4 


12.  Numbering  systems  are  used  accurately  . NA  Y  N 

Errors 

12. A  Numbering  systems  are  inaccurate  (e.g.  ONCE  instead  of 
FIRST,  FIFTEEN  +  HUNDRED  instead  of  ONE-THOUSAND- 

FIVE-HUNDRED)  . NA  0  12  3  4 

12. B  The  palm  orientation  is  inaccurate  (e.g.  height  5'  6"  is 

signed  with  palm  out  instead  of  palm  in)  . NA  0  12  3  4 

12. C  The  movements  are  inaccurate  (e.g.  1991  moves  inward 

toward  mid-line  of  body  instead  of  from  mid-line  outward)  ....NA  0  12  3  4 
12.D  The  locations  of  the  numbers  are  inaccurate  (e.g.  a  person 
taller  than  the  signer  is  7'  tall  and  the  sign  is  made  at  chest 
level  instead  of  above  the  head)  . NA  0  12  3  4 
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13.  DEF:  Numbers  are  often  incorporated  with  signs.  Rather  than  following  the  English 
form  and  signing  TWO  +  THEM  +  GO-TO  (double  index  fingers)  +  STORE,  both  the 
pronoun  and  the  verb  can  be  altered  to  provide  the  same  meaning  such  as  THOSE-TWO  + 
GO-TO  (using  the  number  two  moving  in  the  direction  of  the  store)  +  STORE.  This  latter 
example  uses  the  number  incorporation  feature  of  ASL,  which  is  different  from  the 
linearity  of  English.  Numbers  dealing  with  money,  the  calendar,  age  and  time/o'clock 
also  use  the  incorporation  of  numbers.  Rather  than  signing  FIVE  +  DOLLARS  it  is  signed 
FIVE-DOLLARS.  FOUR  +  WEEKS  +  AGO  is  combined  into  one  sign  FOUR-WEEKS-AGO. 


14.  DEF:  Non-specific  number  signs  are  those  signs  which  indicate  a  number  more  than 
one  without  stating  an  exact  amount  such  as  five  or  eighteen.  The  general  number  of  how 
many  people  or  things  involved  in  the  utterance  (e.g.  many  or  few)  should  be  accurately 
interpreted. 
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13.  Incorporation  of  numbers  is  accurate  . NA  Y  N 

Errors 

13. A  Does  not  incorporate  numbers  in  pronouns  (e.g.  WE  + 

TWO  instead  of  WE-TWO)  . NA  0  12  3  4 

13.B  Does  not  incorporate  numbers  in  verbs  (e.g.  WE-TWO- 

GO-TO  +  STORE)  . NA  0  12  3  4 

13. C  Money/currency  is  signed  inaccurately  (e.g.  FIVE-CENT)  . NA  0  12  3  4 

13. D  Calendar  events  are  signed  inaccurately  (e.g.  THREE- 

WEEKS)  . NA  0  12  3  4 

13. E  Age  is  signed  inaccurately  (e.g.  AGE-FIVE)  . NA  0  12  3  4 

13. F  Time/o'clock  is  signed  inaccurately  (e.g.  TIME-EIGHT, 

FIVE-MINUTES)  . NA  0  12  3  4 


14.  Non-specific  number  signs  are  accurate  (e.g.  FEW,  MANY)  . NA  Y  N 

Error 

14. A  Non-specific  number  signs  are  inaccurate  (e.g.  FEW 

instead  of  SEVERAL)  . NA  0  12  3  4 
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15.  DEF:  The  numbers  are  unclear  if  they  are  not  readable  or  difficult  to  read.  If 
numbers  are  clear,  movements  are  complete  with  no  extraneous  movements  present. 
There  should  be  no  visible  bouncing  or  'stuttering'.  Numbers  should  flow  clearly  from 
one  number  to  the  next.  For  example,  number  79  should  not  move  from  SEVEN  to  FIVE  to 
NINE.  It  should  move  directly  from  the  number  SEVEN  to  NINE. 


16.  DEF:  Numbers  are  formed  with  an  even  rhythm.  Pauses  can  be  used  within  a  series 
of  numbers  but  these  should  be  equal  in  duration  and  follow  the  rules  of  the  appropriate 
numbering  system.  The  pauses  should  have  equal  lengths  of  time  between  them.  If  there 
are  unequal  pauses  within  or  between  numbers  it  can  skew  the  information.  Pauses 
between  series  of  numbers  are  longer  in  duration  than  pauses  within  a  series  of 
numbers.  For  example,  when  discussing  the  height  of  two  children,  4'  6"  and  5'  0", 
there  should  be  a  pause  between  the  two  heights.  It  is  inaccurate  to  have  equal  pauses 
between  all  four  numbers. 
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15.  Numbers  are  clearly  presented  . NA  Y  N 

Errors 

15.A  Numbers  are  not  readable  . NA  0  12  3  4 

15.B  Extraneous  movements  are  present  . NA  0  12  3  4 


16.  The  pace  is  uniform  within  and  between  numbers  . NA  Y  N 

Errors 

16.A  Pauses  between  series  of  numbers  are  not  present,  thus 

appearing  to  be  all  one  number  (e.g.  EIGHT-FOUR-EIGHTEEN 

instead  of  EIGHT-pause-FOUR-pause-EIGHTEEN)  . NA  0  12  3  4 

16. B  Pauses  in  the  middle  of  numbers  are  inaccurate  (e.g.  ONE- 
HUNDRED-pause-NINE-FORTY,  thus  appearing  to  be  two 
separate  numbers)  . NA  0  12  3  4 
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17.  DEF:  The  interpreter's  signing  speed  of  numbers  should  depend  on  the  addressee's 
skill  and/or  preference.  If  the  deaf  consumer  is  a  native  or  near-native  signer,  the 
speed  of  execution  is  less  important.  Fast  production  of  numbers  is  acceptable  if  it  is 
readable.  If  novel  information  is  conveyed  using  the  same  rate  of  speed  that  is  used  with 
familiar  and  commonly  used  numbers,  this  would  be  considered  too  fast.  Slow  signing 
can  annoy  consumers  or  appear  patronizing.  Signing  numbers  slowly  is  appropriate 
when  importance  or  emphasis  is  shown.  For  example,  if  a  person  owes  a  debt  of 
$437.00,  this  is  important  as  compared  to  the  government's  debt  of  $55,700,000 
(unless  of  course,  you  are  talking  to  the  treasurer,  then  this  becomes  important). 
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1  7.  Numbers  are  signed  at  a  readable  speed  . NA  Y  N 

Errors 

17. A  Numbers  are  signed  too  fast  . NA  0  12  3  4 

17. B  Numbers  are  signed  distractingly  slow  . NA  0  12  3  4 


Do  not  copy 


For  research  purposes  only  © 


M.  Taylor 


June  1993 


151 


VOCABULARY 


Rating  Key 


NA  =  Not  applicable,  not  in  the  target  language,  ASL 

Y  =  Yes,  the  skill  was  performed  accurately  100%  of 

the  time 

N  =  No,  the  skill  was  not  performed  accurately  100% 

of  the  time 


m 

O 

1 

2 

3 

4 


Not  applicable,  not  in  the  target  language,  ASL 
No  errors  observed 


Low  frequency, 
High  frequency, 
Low  frequency, 
High  frequency  , 


low  severity 
low  severity 
high  severity 
high  severity 
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VOCABULARY 

18.  DEF:  The  focus  of  this  item  is  to  convey  the  meaning  of  the  utterance  with  accurate 
vocabulary  choices.  The  signs  chosen  must  match  the  meaning  (semantics)  between  the 
source  and  target  languages.  For  example,  in  English  the  word  'disappeared'  is  used  in  a 
variety  of  ways,  such  as,  "My  purse  disappeared"  and  "You  disappeared  after  class."  In 
ASL  these  two  meanings  of  'disappear'  are  signed  differently.  The  nonmanual  grammar 
must  agree  with  the  message.  Compound  signs  are  not  signed  as  if  they  are  two  different 
signs  but  instead  a  sign  with  two  movements  and  two  handshapes  done  in  rapid 
succession.  Fingerspelled  loan  signs  are  signs  that  have  evolved  from  fingerspelling. 

For  accurate  execution  they  have  specific  movements,  handshapes  and  locations,  for 
example,  BACK  and  STYLE.  Also,  noun-verb  pairs  are  signed  differently  from  one 
another.  Nouns  in  this  category  are  signed  with  a  static  repeated  movement  while  verbs 
are  signed  with  a  smooth  often  singular  movement.  Initialized  signs  are  signs  indicating 
the  first  initial  of  the  English  word  used  by  the  speaker  during  the  production  of  the 
sign.  When  providing  an  ASL  interpretation  it  is  inaccurate  to  use  such  signs,  unless, 
the  deaf  consumer  and  the  interpreter  have  previously  agreed  upon  them.  Typically, 
signs  from  any  English  sign  system  should  not  be  used.  Also  when  signing  in  English,  not 
ASL,  the  signed  message  can  have  English  word  endings  such  as  -ing,  -ed,  -ment.  These 
are  not  accurate  in  ASL. 
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VOCABULARY 

18.  ASL  vocabulary  (lexicon)  is  accurate  . NA  Y  N 

Errors 

18. A  Semantics  are  inaccurate  (e.g.  'USE'  as  opposed  to  'USED- 

T0')  . NA  0  12  3  4 

18.B  Nonmanual  facial  grammar  is  inaccurate  . NA  0  12  3  4 

18.C  Compound  signs  are  signed  inaccurately  (e.g.  BLACK- 

NAME)  . NA  0  12  3  4 

18.D  Loan  signs  are  signed  inaccurately  . NA  0  12  3  4 

18. E  Noun-verb  pairs  are  inaccurate  (AIRPLANE  and  FLY, 

CHAIR  and  SIT)  . NA  0  12  3  4 

18. F  Initialized  signs  are  used  (e.g.  IS,  SOCIETY  with  'S' 

handshapes  like  CLASS)  . NA  0  12  3  4 

18. G  English  endings  are  used  (e.g.  -ed,  -ing,  -ment)  . NA  0  12  3  4 
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19.  DEF:  Signs  must  be  clearly  formed  for  ease  of  understanding.  The  handshape  and 
movement  of  the  signs  must  be  accurate.  If  they  are  unclear  or  cause  confusion  they  are 
inaccurate.  The  position  of  the  sign  should  be  in  an  accurate  location.  For  example,  in  a 
formal  setting,  the  signs  that  belong  on  the  forehead  (e.g.  KNOW)  should  occur  there. 
Location,  handshape,  movement,  and  ‘palm  orientation  should  be  accurate.  It  is  deemed 
inaccurate  when  any  potential  for  misunderstanding  occurs.  Signs  should  be  signed 
completely  without  slurring,  that  is,  signing  one  sign  to  the  next  without  definition 
between  the  signs,  thus  appearing  to  be  all  one.  Extraneous  movements,  such  as 
excessive  bouncing  or  repetition  of  the  sign,  should  not  occur.  The  signs  must  be  easily 
understood  and  accurate  nonmanual  facial  grammar  should  accompany  the  signs.  If  signs 
are  two-handed,  two  hands  should  be  used  instead  of  one  unless  this  modification  agrees 
with  the  speaker's  message. 


‘Note:  At  the  time  of  printing,  the  issue  of  palm  orientation  was  controversial. 
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19.  Signs  are  formed  accurately  . NA  y  n 

Errors 

19.A  Handshapes  of  the  signs  are  inaccurate  (e.g.  THEY  instead 

of  THEIR)  . NA  0  12  3  4 

19. B  Movements  of  the  signs  are  inaccurate  (e.g.  signing  WORK 

with  the  movement  of  BUSY)  . NA  0  12  3  4 

19.C  Locations  of  the  signs  are  inaccurate  (e.g.  DOCTOR  signed 

on  the  back  of  the  wrist  could  be  confused  with  DUTY)  . NA  0  12  3  4 

19.D  Palm  orientations  of  the  signs  are  inaccurate  (LAY-ON- 

BACK  instead  of  LAY-ON-STOMACH)  . NA  0  12  3  4 

19. E  Signs  are  not  formed  completely  . NA  0  12  3  4 

19. F  Extraneous  movements  within  signs  are  present  . NA  0  12  3  4 

19.G  Nonmanual  facial  grammar  is  inaccurate  . NA  0  12  3  4 

19. H  Two-handed  signs  are  signed  with  only  one  hand  (e.g. 

HAVE)  . NA  0  12  3  4 
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20.  DEF:  Although  ASL  vocabulary  is  not  as  diverse  as  English  vocabulary,  the 
interpretation  should  attempt  to  be  as  diverse  as  the  English  speaker.  The  message 
should  not  be  repetitive  unless  the  speaker  is  repetitive. 


21.  DEF:  ASL  is  an  evolving  language.  The  interpretation  should  reflect  this  evolution 
and  match  the  needs  of  the  addressee.  Vocabulary  is  ever  changing  in  both  English  and 
ASL.  World  events  affect  the  language,  for  example,  wars  (Persian  Gulf  War),  newly 
elected  public  officials,  and  technology  (microwaves,  computers,  satellite  dishes).  New 
terminology  is  incorporated  in  the  languages  and  should  be  reflected  in  the 
interpretation.  The  ASL  vocabulary  should  match  the  age  of  the  addressee  as  well  as  the 
region  the  addressee  is  from.  For  example,  if  the  addressee  is  from  Vancouver  and 
visiting  Halifax  for  a  meeting,  the  interpretation  should  not  include  local  name  signs  of 
people  and  places,  unless  the  addressee  is  already  familiar  with  them.  They  should  only 
be  used  after  the  meaning  of  the  name  signs  are  clearly  identified. 
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20.  A  diverse  range  of  ASL  vocabulary  is  used  . NA  Y  N 

Errors 

20.A  ASL  vocabulary  is  limited  . NA  0  12  3  4 

20. B  ASL  vocabulary  is  repetitive  (e.g.  FINISH,  WOW,  TRUE)  . NA  0  12  3  4 


21  .  Vocabulary  appropriate  to  the  setting  and  the  consumer  is  used 

(e.g.  MICROWAVE,  a  newly  elected  prime  minister)  . NA  Y  N 

Errors 

21. A  The  vocabulary  is  not  appropriate  for  the  age  of  the 

consumer  . NA  0  12  3  4 

21. B  Regional  vocabulary  is  used  inaccurately  (e.g.  signing 
Edmonton  Eskimos,  a  football  team,  as  Eskimos  in 

Edmonton)  . NA  0  12  3  4 
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REMINDERS 


-Be  sure  to  refer  to  the  work  or  the  interpretation,  not  the 

interpreter,  when  writing  comments.  Avoid  using  'you'.  for 
example,  "you  signed..."  or  "your  interpretation..."  Instead  use 
"the  interpretation  demonstrated  signs  that..."  or  the 
"interpretation  was..." 

-Be  sure  to  write  both  positive  and  negative  comments  when 
providing  feedback  about  the  interpretation. 

-View  the  tape  at  least  twice  without  sound. 

-Be  sure  to  rate  every  skill  and  every  error. 

-If  the  skill  is  not  present,  then  it  is  marked  NA,  not  applicable,  and 
all  of  the  errors  related  to  this  skill  are  also  marked  NA. 

-Remember  to  rate  the  errors  associated  with  the  skill  it  defines 

only.  The  errors  should  not  be  rated  as  if  they  were  associated 
with  other  skills  For  example,  the  error  5.B  "rhythm  or 
cadence  is  inaccurate"  should  be  rated  in  association  with 
skill  5.  "Fingerspelling  is  used  for  emphasis  accurately".  It 
should  not  be  rated  for  overall  rhythm  or  cadence  of  the 
fingerspelling  in  the  interpretation. 
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CLASSIFIERS  AND  SASSes 


Rating  Key 

m 

Not  applicable,  not  in  the  target  language,  ASL 

Y 

Yes,  the  skill  was  performed  accurately  100%  of 

the  time 

N 

No,  the  skill  was  not  performed  accurately  100% 

of  the  time 

NA 

Not  applicable,  not  in  the  target  language,  ASL 

O 

No  errors  observed 

1 

Low  frequency.  low  severity 

2 

Hiah  freauency.  low  severity 

3 

Low  freauencv.  hiah  severity 

4 

Hiah  frequency  .  hiqh  severity 
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CLASSIFIERS  AND  SASSes 

22.  DEF:  Classifiers  can  be  used  as  pronouns  and  verbs.  They  are  an  integral  part  of 
ASL  and,  depending  on  the  information,  should  be  used  in  the  interpretation.  Accurate 
nonmanual  facial  grammar  should  be  used.  If  it  is  not  present  the  target  language  is 
inaccurate.  Classifiers  should  have  both  accurate  handshapes  and  accurate  nonmanual 
grammatical  features. 


23.  DEF:  SASSes  are  size  and  shape  specifiers.  They  visually  display  the  size,  shape, 
depth  and/or  texture  of  an  object  or  location.  To  provide  precise  information,  SASSes 
must  be  used  and  used  accurately.  For  example,  if  the  utterance  is  about  the  buttons  on  a 
blouse,  an  F-HAND  would  likely  be  accurate  rather  than  an  O-HAND  that  would  indicate 
holes  in  the  blouse  or  the  body,  not  buttons.  Also  in  many  instances,  nonmanual 
grammatical  features  must  be  present.  If  they  are  not  present  then  there  is  an  error. 
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CLASSIFIERS  AND  SASSes 

22.  Classifiers  are  used  accurately  . NA  Y  N 

Errors 

22. A  Handshapes  used  for  classifiers  are  inaccurate  (e.g. 

CLASSIFIER-INDEX  instead  of  CLASSIFIER-FLAT-HAND)  . NA  0  12  3  4 

22. B  Nonmanual  facial  grammar  is  inaccurate  . NA  0  12  3  4 


23.  SASSes  (size  and  shape  specifiers)  are  used  inaccurately  . NA  Y  N 

Errors 

23.A  Handshapes  used  for  SASSes  are  inaccurate  (e.g.  F-HAND 

instead  of  O-HAND)  . NA  0  12  3  4 

23. B  Nonmanual  facial  grammar  is  inaccurate  . NA  0  12  3  4 
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24.  DEF:  Classifiers  identify  the  spatial  location  of  a  particular  person  or  thing.  That 
is,  they  can  function  as  verbs  and  locatives.  If  the  classifier  moves  it  must  move 
accurately.  The  classifier  should  clearly  indicate  the  location  and  movement  of  the 
person  or  thing,  such  as  if  the  persons  or  things  are  sitting  in  rows,  a  semi-circle,  or 
haphazardly  moving  about. 

At  times,  an  interpretation  is  accurate  when  just  one  classifier  is  used  but 
becomes  inaccurate  when  more  than  one  classifier  is  used.  The  ideal  interpretation 
accurately  conveys  meaning  using  one  or  several  classifiers.  For  example,  the  spatial 
relationship  of  the  person,  car  and  animal  should  be  accurate  when  signing  CLASSIFIER- 
PERSON,  CLASSIFIER-VEHICLE,  CLASSIFIER-ANIMAL. 
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24.  Classifiers  are  structurally  accurate  . NA  Y  N 

Errors 

24. A  Classifiers  are  structurally  inaccurate  (e.g.  when  a  log  or 
pencil  is  close,  using  the  same  CLASSIFIER-INDEX  rather 
than  CLASSIFIER-INDEX  using  the  forearm  with  the  index 

finger)  . NA  0  12  3  4 

24. B  The  action  of  the  verb  incorporated  with  the  classifier  is 
inaccurate  (e.g.  C-A-R  +  CLASSIFIER-VEHICLE  +  DRIVE- 
DOWN-INCLINE  instead  of  C-A-R  +  CLASSIFIER-VEHICLE- 

DRIVE-DOWN-INCLINE)  . NA  0  12  3  4 

24. C  Inaccurately  identifies  spatial  locations  using  classifiers 

(e.g.  lined  up  in  rows  haphazardly)  . NA  0  12  3  4 

24. D  When  more  than  one  classifier  is  used,  the  relationships 
are  inaccurate  (e.g.  One  hand  is  not  held  constant  while 
signing  the  relationship  of  each  topic  or  action.)  . NA  0  12  3  4 
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25.  DEF:  Classifiers  are  an  important  part  of  ASL.  In  many  interpretations  classifiers 
should  be  present  in  the  target  message.  Because  ASL  is  a  visual  language  and  classifiers 
are  a  visual  way  to  represent  information  and  events,  they  should  be  used  as  much  as 
possible.  It  can  be  tempting  to  sign  many  signs  for  a  particular  message  (linear 
representation),  but  if  it  can  be  expressed  using  classifiers,  then  it  should  be  done  this 
way.  If  the  interpretation  does  not  take  advantage  of  the  spatial  properties  of  ASL,  it 
leaves  the  'interpretation'  up  to  the  deaf  person. 


26.  DEF:  Pictorial  outlining  is  a  specific  way  that  SASSes  can  be  used.  It  is  a  method  of 
signing  that  shows  the  shape  or  outline  of  something.  It  conveys  information  and 
provides  specific  visual  meaning.  Often  it  is  used  in  conjunction  with  classifiers  and 
SASSes.  The  size  and  shape  of  the  object  must  be  accurate.  For  example,  if  describing 
the  shape  of  Vancouver  Island,  one  might  choose  to  outline  the  shape  of  the  island.  If  one 
does  not  know  the  shape  of  the  island  it  can  not  be  done  accurately  unless  the  details  of 
the  shape  are  present  in  the  source  language.  When  the  shape  of  something  is 
symmetrical,  both  hands  should  move  in  the  same  manner  simultaneously. 
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25.  Classifiers  are  used  extensively  . NA  Y  N 

Error 

25.A  Individual  signs  are  used  instead  of  classifiers  (e.g.  CAR  + 

HIT  +  MAN  instead  of  MAN  +  CLASSIFIER-INDEX  +  C-A-R 

+  CLASSIFIER-VEHICLE-HIT-CLASSIFIER-INDEX)  . NA  0  12  3  4 

25. B  Few  classifiers  are  used  in  comparison  to  what  could  be 

used  . NA  0  12  3  4 


26.  Pictorial  outlining  is  accurate  . NA  Y  N 

Errors 

26. A  Sizes  are  inaccurate  (e.g.  a  postage  stamp  instead  of  a  3X5 

card)  . NA  0  12  3  4 

26. B  Shapes  are  inaccurate  . NA  0  12  3  4 

26. C  One  finger  is  not  held  constant  while  the  other  finger  is 

outlining  an  unusual  shape  . NA  0  12  3  4 

26. D  Inaccurate  when  both  hands  should  move  simultaneously 
and  don't  (e.g.  OUTLINE-CARD,  both  hands  should  move 

simultaneously,  not  consecutively)  . NA  0  12  3  4 
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27.  DEF:  Because  classifiers  often  function  as  pronouns,  the  referent  noun  needs  to  be 
identified  prior  to  using  a  classifier.  It  is  inaccurate  to  sign  a  classifier  without  first 
identifying  what  the  classifier  represents.  By  establishing  the  noun  first,  the  addressee 
knows  what  the  pronoun  refers  to.  Without  this  prior  knowledge  the  pronoun  is  just 
that,  a  pronoun  without  a  referent,  and  is  therefore  inaccurate. 


28.  DEF:  SASSes  are  a  description  of  something.  Before  the  description  is  rendered  the 
interpretation  must  identify  the  referent  noun  or  the  pronoun  before  describing  it.  For 
example,  if  a  person  is  talking  about  the  waves  on  the  lake,  the  lake  must  be  identified 
prior  to  describing  the  waves  using  SASSes. 
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27.  The  noun  is  identified  prior  to  using  a  classifier  (e.g.  pencil  then 

classifier)  . NA  Y  N 

Errors 

27. A  The  noun  is  identified  after  using  a  classifier  (e.g. 

CLASSIFIER-INDEX  +  PENCIL)  . NA  0  12  3  4 

27. B  The  noun  is  not  identified  at  all  (e.g.  CLASSIFIER-VEHICLE 

+  BREAKDOWN  without  indicating  what  vehicle)  . NA  0  12  3  4 


28.  SASSes  are  signed  after  the  noun  or  pronoun  (e.g.  T-U-B-E  + 

F-HANDSHAPE-THIN-LONG)  . NA  Y  N 

Error 

28. A  SASSes  are  signed  before  the  noun  or  pronoun  is  identified 

(e.g.  THIN-LONG  +  T-U-B-E  instead  of  T-U-B-E  first)  . NA  0  12  3  4 
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STRUCTURING  SPACE 


Rating  Key 


NA  =  Not  applicable,  not  in  the  target  language,  ASL 

Y  =  Yes,  the  skill  was  performed  accurately  100%  of 

the  time 

N  =  No,  the  skill  was  not  performed  accurately  100% 

of  the  time 


m 

O 

1 

2 

3 

4 


Not  applicable,  not  in  the  target  language,  ASL 
No  errors  observed 


Low  frequency, 
High  frequency, 
Low  frequency, 
High  frequency  , 


low  severity 
low  severity 
high  severity 
high  severity 
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STRUCTURING  SPACE 

29.  DEF:  The  spatial  relationship  must  be  accurate  when  more  than  one  person,  place 
or  thing  is  mentioned.  For  example,  "Paul,  who  is  38,  and  Mark,  who  is  36,  live  in 
Washington  State"  could  be  signed  with  the  name  of  each  person  in  a  different  location 
and  the  corresponding  age  signed  in  locations  corresponding  to  each  man.  Structuring 
space  should  be  accurate  in  relation  to  the  actual  location  of  people,  places  or  things 
referred  to  in  the  utterance.  For  example,  if  Joe  is  on  the  left  and  George  on  the  right 
and  both  are  present  in  the  room,  their  actual  spatial  location  is  used  to  identify  them. 
In  ASL  if  there  are  actual  situations,  people,  or  places  discussed,  these  locations  are 
established  corresponding  to  real  life.  If  talking  about  cities  in  Canada  and  the  United 
States,  Helena,  Montana  should  be  'below'  Calgary  and  Edmonton,  Alberta.  Edmonton 
should  be  above  Calgary.  They  should  all  be  equal  distance  from  one  another  to  agree 
with  their  relative  geographic  relationship. 
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STRUCTURING  SPACF 

29.  Structuring  space  is  accurate  . NA  y  n 

Errors 

29. A  Spatial  relationships  are  inaccurate  (e.g.  They  stood  in 
opposite  corners  of  the  room',  is  signed  with  both  in  the 

same  corner)  .  0  12  3  4 

29. B  Spatial  agreement  is  inaccurate  (e.g.  'Eva  goes  to 

preschool  daily’  is  signed  with  GO-TO  moving  to  a  different 

location  from  where  the  preschool  was  established)  . NA  0  12  3  4 

29. C  The  actual  location  of  people  or  things  is  not  followed  (e.g. 

Sheree  and  Kathy  are  sitting  in  the  room  and  the  pointing  is 
directed  to  a  spatial  location  different  from  where  they  are 

sitting)  . NA  0  12  3  4 

29. D  Inaccurately  uses  space  for  places/geography  (e.g. 

Washington  State  should  be  to  the  left  and  New  York  to  the 

right  of  the  interpreter)  . NA  0  12  3  4 
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30.  DEF:  When  the  speaker  is  talking  about  people  interacting  with  one  another,  the 
interpretation  should  reflect  this.  The  most  basic  interaction  is  that  of  the  narrator 
(the  person  talking)  and  one  other  person.  The  first  person,  that  is  the  speaker, 
addresses  the  second  person,  the  person  spoken  to.  The  body  orientation,  including  the 
head  and  shoulders,  must  have  the  accurate  orientation.  If  Joe  is  tall  and  looking  down  at 
his  toddler  on  the  floor,  then  he  should  be  looking  down  when  he  addresses  his  toddler. 
The  toddler  should  look  up  in  the  direction  of  where  Joe  is  located  in  space.  The  subject 
can  be  indicated  by  "becoming"  a  person  using  eye  gaze  (e.g.  left,  right,  up,  down),  body 
shifts,  facial  expressions,  and  mannerisms  of  that  person,  then  back  to  narrator  or 
speaker  by  looking  at  the  addressee  and  resuming  a  neutral  position.  If  the  conversation 
is  lengthy,  the  interpretation  should  not  include  repetition  of  "FATHER  +  SAY"  and  the 
"TODDLER  +  SAY".  This  repetition  in  English  is  acceptable.  In  ASL,  this  repetition  is 
perceived  as  redundant. 

When  two  people  are  talking  or  several  issues  are  discussed,  the  interpretation 
must  clearly  indicate  who  is  talking  to  whom  and  what  issues  are  being  discussed  by 
whom.  Also,  it  must  be  accurately  conveyed  when  the  speaker  assumes  the  role  of 
narrator  speaking  to  the  addressee  or  the  audience. 
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30.  Assuming  the  role  of  characters  and/or  ideas  is  accurate  . NA  Y  N 

Errors 

30. A  Body  orientation  for  characters  or  ideas  is  inaccurate  (e.g. 
small  boy  talking  to  his  tall  father  or  a  comparison 

between  Italian  and  French  food)  . NA  0  12  3  4 

30. B  Assumes  the  role  of  persons  other  than  the  narrator 

inaccurately  (e.g.  it  is  not  clear  who  is  talking  to  whom)  . NA  0  12  3  4 

30. C  Assumes  the  role  of  narrator  inaccurately  (e.g.  doesn't 

address  the  addressee  as  a  narrator)  . NA  0  12  3  4 

30. D  Indicates  the  name  of  the  person  or  idea  redundantly  (e.g. 

repeatedly  signing  HE  +  SAID  and  SHE  +  SAID)  . NA  0  12  3  4 
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31.  DEF.  Once  locations  are  assigned  in  space  they  should  be  held  constant.  They  should 
not  be  misplaced  or  altered  without  signaling  that  restructuring  is  occurred. 
Restructuring  is  perfectly  acceptable.  However,  it  must  be  accurately  executed  with  a 
clear  indication  in  the  interpretation  that  there  is  a  change  in  spatial  location.  A  new 
referent  can  not  be  placed  in  the  location  of  a  previously  established  referent  without 
first  indicating  this  change.  If  the  change  is  not  indicated,  misunderstanding  can  result. 


32.  If  signs  are  used  without  structuring  space,  this  is  an  error.  For  example,  the 
advantages  and  disadvantages  of  parents  paying  for  their  child's  education  can  be  signed 
in  several  ways.  One  alternative  is  to  sign  the  advantages  near  the  left  side  of  the  body 
and  the  disadvantages  near  the  right  side  of  the  body.  Another  way  is  to  sign  exactly  what 
is  stated  without  setting  up  spatial  locations  for  the  advantages  and  disadvantages.  This 
latter  manner  of  interpretation  does  not  take  advantage  of  ASL's  spatial  properties. 
Individual  signs  are  used  instead  of  structuring  space,  that  is,  strings  of  signs  are 
presented  rather  than  manipulating  space. 

Space  should  be  used  extensively  and  with  ease.  Use  of  the  passive  hand, 
can  demonstrate  the  ease  with  which  space  is  structured.  Both  the  active  and  passive 
hands  should  be  used  throughout  the  interpretation. 
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31  .  Structure  of  space  is  held  constant  . NA  Y  N 

Errors 

31  .A  Locations  of  people  or  things  are  changed  or  misplaced 

inaccurately  (e.g.  Katherine  was  on  the  left  and  is  placed  on 

the  right  inaccurately)  . NA  0  12  3  4 

31. B  Restructuring  space  is  inaccurate  (e.g.  When  a  new  topic 
is  discussed,  it  is  not  clear  that  newly  assigned  spatial 

locations  are  used)  . NA  0  12  3  4 

31. C  Assigning  two  different  referents  to  the  same  location  (e.g. 

the  car  and  the  bus)  . NA  0  12  3  4 


32.  Space  is  used  extensively  . NA  Y  N 

Errors 

32. A  Individual  signs  are  used  instead  of  structuring  space  . NA  0  12  3  4 

32. B  Passive  hand  is  used  for  structuring  space  inaccurately  . NA  0  12  3  4 
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33.  DEF:  Not  only  are  people,  places  and  things  placed  in  space;  verbs  are  manipulated 
in  space  and  must  agree  with  previously  assigned  locations.  If  the  location  for  Joe  is  on 
the  right  side  of  the  interpreter's  space  and  someone  gives  something  to  him,  the 
movement  of  the  verb  should  move  toward  the  right  side  where  Joe  is  established.  If  it 
moves  away  from  Joe,  it  gives  the  wrong  meaning. 

Body-anchored  verbs  are  a  type  of  verb  that  uses  space.  In  this  case,  as  the  name 
implies,  the  signs  are  made  on  the  body.  Examples  of  this  are  SURGERY  and  HIT.  If 
possible,  these  verbs  should  be  articulated  at  the  site  of  the  area  discussed,  for  example, 
the  left  eye,  or  the  index  finger  on  the  right  hand. 

In  addition  to  placing  things  and  people  in  space  and  signing  on  the  body  at  the 
right  location,  nonmanuals  associated  with  the  verb  should  be  used  accurately.  For 
example,  often  the  eyes  should  follow  the  movement  of  the  verb.  For  example,  if  the 
operation  is  on  the  back  of  the  arm,  the  eyes  often  glance  or  look  at  this  area  of  the  body. 
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Signs  verbs  by  modulating  the  use  of  space  accurately  (e.g.  She 

gave  the  money  to  them)  . NA  y  n 

Errors 

33. A  The  verb  is  modulated  to  show  location  inaccurately  (e.g. 

GO-TO-LEFT  should  be  GO-TO-RIGHT)  . NA  0  12  3  4 

33. B  Directional  verbs  showing  subject-object  relationships 
are  inaccurate  (e.g.  ME-BORROW  moving  toward  the 

addressee  instead  of  toward  the  interpreter)  . NA  0  12  3  4 

33. C  Verbs  which  can  be  signed  on  specific  body  locations  are 
inaccurate  (e.g.  body-anchored  verbs  such  as  SHAVE-ON- 

HEAD,  HURT-AT-EAR) .  NA  0  1  2  3  4 

33. D  Adverbial  nonmanual  facial  grammar  is  inaccurate  (e.g. 

'STA'  and  'MM')  . NA  0  12  3  4 
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34.  DEF:  All  languages,  including  ASL,  have  methods  for  articulating  pronominalization; 
who  is  saying  what  to  whom  and  who  is  doing  what  to  whom. 

When  a  referent  is  moved  from  one  location  to  another  it  should  be  done 
accurately  and  should  indicate  from  where  and  to  where  the  referent  is  moved.  If  the 
speaker's  perspective  changes  the  referent  may  change  locations;  it  may  move  closer  or 
farther  away.  The  target  language  should  clearly  indicate  whether  there  is  involvement 
of  one,  several,  many  or  all  (e.g.  single  subject  or  object  or  multiple  subjects  or 
objects).  If  it  is  not  stated  or  is  unclear,  it  is  an  error. 

35.  DEF:  Indexing  is  used  frequently  in  ASL.  It  is  one  way  space  is  used  to  provide 
information  to  the  addressee  in  a  structured  manner.  One  form  of  indexing  is  to  use  the 
passive  hand  as  a  base.  The  fingers  on  the  passive  hand  are  used  as  the  location  from 
where  future  information  referring  to  that  item  may  be  located.  The  hand  must  have  the 
accurate  palm  orientation,  the  accurate  fingers  used  and  accurate  order  of  counting.  For 
example,  if  discussing  four  children  in  the  family,  the  passive  hand  can  be  used  in  a 
FOUR-HANDSHAPE,  with  the  palm  facing  inward.  The  index  finger  tends  to  be  the  oldest 
child  and  the  baby  finger  often  refers  to  the  youngest  child.  By  pointing  to  the  specific 
finger  for  that  child,  these  four  fingers  can  be  used  repeatedly,  providing  information 
pertaining  to  each  child.  The  order  of  counting  must  be  accurate.  That  is,  if  two  topics 
are  discussed  and  two  fingers  are  used  on  the  passive  hand  to  refer  to  these  two  topics, 
the  ones  most  often  used  are  the  index  and  middle  fingers.  If  neutral  space  is  used  for 
indexing,  the  location  of  each  item  or  person  must  be  identified  prior  to  talking  about 
them.  Then  all  of  the  following  information  can  be  expressed  by  pointing  or  indexing 
that  location  in  space.  Unless  the  speaker  is  redundant,  a  variety  of  indexing  techniques 
should  be  used,  not  just  one  (depending  on  the  topic,  of  course). 
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34.  Pronominalization  is  accurate  . NA  Y  N 

Errors 

34 .A  Who  is  saying  what  to  whom  is  inaccurate  . NA  0  12  3  4 

34. B  Who  is  doing  what  to  whom  is  inaccurate  . NA  0  12  3  4 

34. C  When  the  person  or  thing  is  moving,  moves  a  referent 

from  place  to  place  inaccurately  . NA  0  12  3  4 

34. D  When  the  speaker's  perspective  changes,  moves  a  referent 

from  place  to  place  inaccurately  . NA  0  12  3  4 

34. E  Distinctions  among  the  involvement  of  some,  many  or  all 

of  the  people,  places  or  things  are  inaccurate  . NA  0  12  3  4 


35.  Indexing  techniques  are  accurate  . NA  Y  N 

Errors 

35. A  Listing  on  fingers  is  inaccurate  . NA  0  12  3  4 

35. B  Pointing  to  locations  is  inaccurate  . NA  0  12  3  4 

35. C  One  indexing  system  is  used  repeatedly  . NA  0  12  3  4 
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REMINDERS 


-Be  sure  to  refer  to  the  work  or  the  interpretation,  not  the 

interpreter,  when  writing  comments.  Avoid  using  'you',  for 
example,  "you  signed..."  or  "your  interpretation..."  Instead  use 
"the  interpretation  demonstrated  signs  that..."  or  the 
"interpretation  was..." 

-Be  sure  to  write  both  positive  and  negative  comments  when 
providing  feedback  about  the  interpretation. 

-View  the  tape  at  least  twice  without  sound. 

-Be  sure  to  rate  every  skill  and  every  error. 

-If  the  skill  is  not  present,  then  it  is  marked  NA,  not  applicable,  and 
all  of  the  errors  related  to  this  skill  are  also  marked  NA. 

-Remember  to  rate  the  errors  associated  with  the  skill  it  defines 

only.  The  errors  should  not  be  rated  as  if  they  were  associated 
with  other  skills  For  example,  the  error  5.B  "rhythm  or 
cadence  is  inaccurate"  should  be  rated  in  association  with 
skill  5.  "Fingerspelling  is  used  for  emphasis  accurately".  It 
should  not  be  rated  for  overall  rhythm  or  cadence  of  the 
fingerspelling  in  the  interpretation. 
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GRAMMAR 


Rating  Key 


NA  =  Not  applicable,  not  in  the  target  language,  ASL 

Y  =  Yes,  the  skill  was  performed  accurately  100%  of 

the  time 

N  =  No,  the  skill  was  not  performed  accurately  100% 

of  the  time 


NA 

O 

1 

2 

3 

4 


Not  applicable,  not  in 
No  errors  observed 
Low  frequency, 

High  frequency, 

Low  frequency, 

High  frequency  , 


the  target  language,  ASL 

low  severity 
low  severity 
high  severity 
high  severity 
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GRAMMAR 

36.  DEF:  Each  type  of  structure  has  its  own  rules.  Declarative  sentences  are  one  of  the 
most  common  sentence  structures  in  any  language.  For  example,  in  declarative 
statements  (Henry  loves  Edith),  there  is  often  a  nod  at  the  end  of  the  sentence 
accompanied  with  a  tightening  of  closed  lips  to  emphasize  the  statement  is  true.  Sentence 
structures  such  as  commands,  ordering  the  addressee  to  do  something  uses  direct  eye 
contact  with  the  person  being  spoken  to. 

Because  English  is  the  source  language  and  has  different  pausing  patterns  from 
ASL,  problems  with  the  clarity  of  the  target  message  can  occur.  Instead  of  signing  a 
conditional  sentence  with  a  pause  between  the  two  parts  (If  I  go,  I'll  have  to  stop  at  the 
bank),  run-on  sentences  can  occur.  This  contrasts  with  relative  clauses  (My  mother, 
who  is  an  interpreter,  likes  people)  where  there  is  no  pause  in  ASL  but  may  have  a 
pause  in  English. 

Each  structure  should  be  complete  and  there  should  be  a  variety  of  sentence 
structures  without  overuse  of  any  one  particular  structure  (depending  on  the  speaker, 
setting  and  content  of  message,  of  course). 
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GRAMMAR 

36.  Statements  are  signed  accurately  . NA  Y  N 

Errors 

36.A  Nonmanual  facial  grammar  is  inaccurate  . NA  0  12  3  4 

36. B  ASL  pausing  is  inaccurate  . NA  0  12  3  4 

36. C  The  structure  is  incomplete  (e.g.  IF  ME  GO-TO  J-A-M-A- 

l-C-A...and  it  is  not  finished)  . NA  0  12  3  4 

36. D  Statement  sentence  structures  are  used  excessively  (e.g. 

conditionals)  . NA  0  12  3  4 
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37.  DEF:  There  are  different  types  of  question  structures  in  ASL,  such  as  rhetorical 
questions  (Why  do  I  want  to  go?  I  love  traveling),  wh-questions  (Where  do  you  live?) 
and  yes-no  questions  (Did  you  like  the  movie?). 

Nonmanuals  for  these  structures  must  be  accurate.  For  example,  wh-questions 
have  a  brow  squint  throughout  the  sentence,  whereas  the  yes-no  questions  have  brows 
raised.  Without  these  specific  nonmanual  grammatical  signals  confusion  can  occur. 

Rhetorical  questions  are  used  to  draw  attention  to  specific  information  that  will 
be  provided  following  the  rhetorical  question.  This  structure  is  not  eliciting  a  response 
from  the  addressee.  It  is  an  error  if  it  is  used  excessively  because  it  loses  its  impact. 

For  conditional  sentence  structures  ending  in  questions  (If  I  go  to  the  bank  do  I 
need  identification?)  and  for  rhetorical  questions,  pausing  is  important.  For  example, 
in  conditionals  there  is  a  pause  before  the  question  portion  of  the  sentence  and  in 
rhetoricals  the  pause  is  after  the  question.  If  it  is  not  present,  it  is  inaccurate. 

38.  DEF:  Facial  adjectives  and  adverbs  are  grammatical  signals  which  accompany 
signing.  They  are  necessary  to  convey  accurate  information  about  the  specific  meaning 
of  the  utterance.  Meaning  is  lost  without  these  necessary  features.  They  assist  in 
clarifying  meaning  such  as  large,  small  or  average  in  size.  They  can  indicate  width  by 
describing  if  it  is  wide  or  narrow.  They  can  also  identify  the  texture  whether  something 
is  smooth  or  rough.  In  addition  they  describe  how  something  was  accomplished,  for 
example,  slowly  or  quickly. 

Eye  gaze  is  another  grammatical  signal.  For  example,  the  sign  READ  most  often 
has  an  eye  gaze  to  the  location  of  the  signing.  Also,  eye  gaze  can  be  used  to  show 
emphasis,  approximations,  and  uncertainty. 
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37.  Signs  questions  accurately  . NA  Y  N 

Errors 

37. A  Nonmanual  facial  grammar  is  inaccurate  . NA  0  12  3  4 

37. B  ASL  pausing  is  inaccurate  . NA  0  12  3  4 

37. C  The  structure  is  incomplete  (e.g.  WESLEY+LEAVE  without 

adding  WHY,  thus  appearing  to  be  a  yes-no  question)  . NA  0  12  3  4 

37. D  Question  sentence  structures  are  used  excessively  (e.g. 

rhetorical  questions)  . NA  0  12  3  4 


38.  Nonmanual  facial  grammar  is  accurate  . NA  Y  N 

Errors 

38.A  Facial  adverbs  are  inaccurate  (e.g.  "MM"  and  "TH")  . NA  0  12  3  4 

38. B  Facial  adjectives  are  inaccurate  (e.g.  "CHA")  . NA  0  12  3  4 

38. C  Eye  gaze  is  inaccurate  . NA  0  12  3  4 
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39.  DEF:  Pluralization  must  be  made  as  equally  clear  in  the  interpretation  as  it  is  in 
the  source  language  message.  There  are  many  ways  to  convey  pluralization.  This  item  is 
specifically  addressing  the  articulation  of  signs.  In  order  to  express  pluralization  either 
a  plural  indicator  must  be  signed  such  as  FEW  or  MANY,  or  the  sign  itself  must  be 
altered  to  indicate  more  than  one.  For  example,  classifiers  are  often  made  plural  by 
repeating  the  sign  in  different  locations.  For  signs  with  an  agent  sign,  only  the  agent 
sign  is  repeated,  not  the  entire  word  such  as  INTERPRET-AGENT  +  INTERPRET-AGENT. 

In  general,  this  is  not  accurate,  except  in  extremely  formal  settings. 


40.  DEF:  Duration  (e.g.  ALL-DAY),  regularity  (e.g.  EVERY-AFTERNOON),  approximate 
time  (ABOUT-FIVE-O'CLOCK),  the  meaning  "repeated  and  prolonged"  (e.g.  YEARS-AND- 
YEARS),  relationship  to  present  time  (e.g.  RECENTLY,  FUTURE)  are  all  ways  to  express 
the  meaning  of  time.  The  meaning  will  be  skewed  if  these  concepts  are  not  signed 
accurately.  Differences  such  as  afternoon'  and  all  afternoon  must  be  clearly 
differentiated.  This  difference  and  others  like  it  can  be  conveyed,  for  example  by 
altering  the  sign  movement  to  show  the  difference  and  using  nonmanual  facial  markers  to 

convey  the  accurate  meaning. 

Time  indicators  (e.g.  TOMORROW,  LAST-YEAR)  generally  appear  at  the  beginning 
of  an  ASL  utterance. 
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39.  Signs  plurals  accurately  (i.e.  more  than  one)  . NA  Y  N 

Errors 

39. A  Movements  are  inaccurate  . NA  0  12  3  4 

39. B  Repetitions  are  inaccurate  (e.g.  no  repetition  of  the  sign 

thereby  appearing  singular)  . NA  0  12  3  4 

39. C  Plural  indicators  are  inaccurate  (e.g.  MANY)  . NA  0  12  3  4 


40.  Time  is  signed  accurately  (e.g.  EVERY-NIGHT,  RECENT-PAST)  .  NA  Y  N 

Errors 

40.A  Time  is  signed  inaccurately  . NA  0  12  3  4 

40. B  Nonmanual  facial  grammar  is  inaccurate  . NA  0  12  3  4 

40. C  Time  is  signed  in  an  inaccurate  location  in  the  sentence 
structure  (e.g.  at  the  end  of  the  paragraph  instead  of  the 
beginning)  . NA  0  12  3  4 
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41.  DEF:  Temporal  aspect  (e.g.  SLEEPS-UNUSUALLY-OFTEN,  LAUGHS-LONG-TIME)  is 
a  way  in  which  some  signs,  often  verbs,  are  modulated  to  show  additional  meaning  such 
as  regularity  and  length  of  the  occurrence.  It  is  not  specific  (e.g.  ten  times,  two  hours) 
rather,  it  is  general  (e.g.  many  times,  long  time)  in  meaning.  It  indicates  the 
interpretation  of  the  speaker's  perception  and  feeling  of  the  meaning  of  the  verb. 
Temporal  aspect  is  associated  with  how  something  was  done  over  time  and  for  how  long  it 
occurred.  Was  it  regular  (e.g.  GO-TO  +  GO-TO  +  SCHOOL)  or  infrequent?  Was  it  done 
repeatedly  (e.g.  WORK-OVER-AND-OVER-AGAIN)?  It  is  important  to  use  temporal 
aspect  instead  of  signing  "I  +  GO  +  THERE  +  OFTEN".  Specific  nonmanual  markers  are 
often  associated  with  temporal  aspect.  For  example,  WORK-OVER-AND-OVER-AGAIN, 
the  nonmanual  of  'STA'  is  often  present.  This  facial  grammar  must  agree  with  the 
production  and  meaning  of  the  sign. 
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41  .  Temporal  aspect  is  accurate  (e.g.  regularly,  for  a  long  time)  .  NA  Y  N 

Errors 

41. A  Temporal  aspect  is  inaccurate  . NA  0  12  3  4 

41. B  Nonmanual  facial  grammar  is  inaccurate  . NA  0  12  3  4 
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INTERPRETING 


Rating  Key 


NA  =  Not  applicable,  not  in  the  target  language,  ASL 

Y  =  Yes,  the  skill  was  performed  accurately  100%  of 

the  time 

N  =  No,  the  skill  was  not  performed  accurately  100% 

of  the  time 


NA 

O 

1 

2 

3 

4 


Not  applicable,  not  in  the  target  language,  ASL 
No  errors  observed 

low  severity 
low  severity 
high  severity 
high  severity 


Low  frequency, 
High  frequency, 
Low  frequency, 
High  frequency  , 
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INTERPRETING 


42.  DEF:  The  source  message  should  be  accurately  expressed  both  linguistically  and 
culturally  to  have  equivalency  of  meaning.  If  any  meaning  is  deleted  or  added  it  is 
inaccurate.  This  does  not  mean  that  a  cultural  adjustment  is  an  error  if  it  is  used  (e.g. 
substitutions,  additions)  to  convey  the  meaning.  If  the  cultural  adjustments  equate  the 
meaning,  without  adding  to  or  subtracting  from  it,  then  it  is  accurate.  For  example,  if 
the  speaker  is  talking  about  the  annoying  sound  outside,  the  interpretation  must  provide 
information  about  the  sound.  The  interpretation  may  include  not  only  that  there  is 
sound,  but  also,  what  does  it  sound  like  (e.g.  a  hammer,  someone  walking  or  an  electric 
egg  beater). 
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INTERPRETING 


42.  The  meaning  of  the  source  language  (English)  is  interpreted 

accurately  . NA  Y  N 

Errors 

42.A  Addition  of  meaning  occurs  . NA  0  12  3  4 

42. B  Omission  of  meaning  occurs  other  than  numbers  and 
fingerspelling  (e.g.  incomplete  meaning,  nuances  are 
omitted,  opposite  meaning  is  conveyed  such  as  omitting 

negation,  thus  making  the  interpretation  look  affirmative)  NA  0  12  3  4 

42. C  Cultural  adjustments  are  made  inaccurately  . NA  0  12  3  4 
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43.  DEF:  The  structure  of  the  target  language  should  be  in  ASL.  It  should  not  follow 
English  word  order  and  should  have  little  to  no  influence  from  English.  If  English 
influences  are  present,  such  as  initialized  signs  (for  example,  REHABILITATION  signed 
like  HELP  but  with  the  letter  'R'  instead  of  the  A-handshape),  this  can  be  confusing  to 
the  addressee  who  may  not  be  sure  of  the  meaning  or  may  have  seen  the  same  sign  used  to 
mean  "resources".  This  skews  the  ASL  meaning  and  requires  the  addressee  to  interpret 
the  meaning.  Note:  Initialized  signs  do  not  include  old  signs  such  as  WORLD,  which  is  not 
an  invented  English  sign  developed  for  the  purpose  of  teaching  English  to  children.  Older 
signs  have  been  used  for  generations  and  have  changed  over  time  as  any  language  changes 
(e.g.  RESPONSIBLE  and  LANGUAGE  which  changed  to  initialized  signs  many  years  prior  to 
the  creation  of  English  sign  systems).  The  grammatical  pauses  should  be  those  of  ASL, 
not  English.  English  mouthing  should  not  be  evident,  if  it  skews  the  nonmanual 
grammatical  markers. 


44.  DEF:  It  is  important  to  match  and  convey  the  same  intent  as  the  speaker's.  The 
intent  includes  expression  (e.g.  intensity,  emphasis)  and  affect  (e.g.  mood).  The 
interpretation  should  not  have  facial  expression  that  is  incongruent  with  the  intent  of 
the  speaker.  The  facial  expression  accurately  reflect  those  aspects  of  the  speakers  mood 

and  delivery. 


Do  not  copy 


For  research  purposes  only  © 


M.  Taylor 


June  1993 


193 


43.  The  target  language  interpretation  successfully  avoids 

interference  from  the  source  language  . NA  Y  N 

Errors 

43. A  The  interpretation  follows  English  structure,  not  ASL  (e.g. 

word  for  word,  sign  for  sign)  . NA  0  12  3  4 

43. B  Initialized  signs  are  used  (e.g  DEVELOP  [with 

initialization  *D']  +  MENT)  . NA  0  12  3  4 

43. C  Grammatical  pauses  are  inaccurate  . NA  0  12  3  4 

43. D  English  is  mouthed  when  it  should  not  be  present  . NA  0  12  3  4 

43. E  Nonmanual  grammar  is  inaccurate . NA  0  12  3  4 


44.  The  intention  of  the  speaker  is  reflected  accurately  . NA  Y  N 

Errors 

44. A  The  affect  used  in  the  interpretation  is  inaccurate  (e.g. 

does  not  match  the  mood  of  the  speaker)  . NA  0  12  3  4 

44. B  The  interpretation  is  inaccurately  monotone  (e.g.  lack  of 
facial  expression,  intensity  or  stress  of  signing  is  demure) 

. NA  0  12  3  4 

44. C  The  interpretation  is  inaccurately  expressive  (e.g. 

exaggerated  mouthing,  intensity  or  stress  of  signing  is 

excessive)  . NA  0  12  3  4 
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45.  DEF:  The  interpretation  should  match  the  speaker's  register.  If  the  setting  and  the 
speaker's  affect  is  formal  then  the  interpretation  should  be  formal.  If  within  this 
formal  atmosphere  the  speaker  changes,  whether  for  a  short  or  long  period,  the 
interpretation  should  accurately  reflect  these  register  changes.  English  register  is 
matched  in  ASL  in  a  variety  of  ways.  For  example,  large  signing  for  formal  or  platform 
speeches,  and  smaller  signing  for  more  intimate  one  on  one  settings  is  often  used. 
Vocabulary  is  also  an  indication  of  register.  For  example,  specific  name  signs  not  known 
by  all  the  addressees  could  be  intimate  register,  whereas  formal  vocabulary,  are  those 
vocabulary  items  which  are  most  likely  known  by  all  addressees  and  do  not  require  an 
intimate  knowledge  of  the  discussion/text  to  understand  the  meaning.  At  times  the 
interpretation  can  reflect  an  informal  or  casual  register  by  using  excessive  or 
exaggerated  body  shifts,  that  is,  what  often  is  used  for  children.  The  interpretation 
should  reflect  the  register  of  the  speaker  and  should  only  change  register  when  the 
speaker  changes. 
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The  ASL  register  reflects  the  English  register  accurately  . NA  Y  N 

Errors 

45. A  The  register  is  inaccurately  formal  . NA  0  12  3  4 

45. B  The  register  is  inaccurately  informal  (e.g.  KNOW  signed 

on  the  cheek  instead  of  the  forehead)  . NA  0  12  3  4 

45. C  The  register  is  inaccurately  intimate  (e.g.  using  specific 

vocabulary  the  addressee  is  likely  not  to  know)  . NA  0  12  3  4 

45. D  Maintains  the  same  register  when  the  speaker  changes 
his/her  register  inaccurately  (e.g.  from  a  formal  lecture 
to  a  group  of  individuals  changing  to  a  very  informal  one  on 
one  discussion  with  a  participant  during  a  demonstration  of 

counseling  techniques)  . NA  0  12  3  4 

45. E  Inaccurately  changes  the  register  when  the  speaker  does 

not  . NA  0  12  3  4 
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46.  DEF:  There  are  a  variety  of  ways  to  show  different  perspectives  of  an  event  or 
action.  One  way  is  the  choice  of  classifiers  and  SASSes.  They  are  used  to  express  not 
only  the  size,  shape  and  location  of  something  but  also  they  are  used  to  convey  the 
speaker's  vantage  point  on  the  subject.  For  example,  if  a  person  is  walking  close  to  the 
water  on  the  beach  and  a  large  wave  comes  up,  the  interpretation  would  look  different 
than  if  someone  was  watching  the  large  waves  crashing  on  the  beach  from  a  hotel  room. 
The  focus  of  the  action  (the  closing  of  the  door  or  the  noise  the  slam  made  when  it  closed) 
is  combined  with  the  location  of  the  event  in  terms  of  its  distance  or  closeness  to  the 
action  or  event. 


47.  DEF:  It  is  natural  for  errors  to  occur  while  interpreting.  Once  errors  are 
identified  they  should  be  repaired  accurately  and  repaired  with  ease.  If  they  are 
repaired  with  apologies  (e.g.  SORRY  or  EXCUSE  ME)  this  would  be  considered  an 
awkward  repair.  If  information  is  missed  or  not  understood,  this  should  be  reflected  in 
the  interpretation  and  is  not  considered  an  error  within  this  category.  Excessive 
repairing  should  be  avoided.  If  errors  are  repaired  excessively  it  means  that  a 
distraction  occurs  causing  the  viewer,  in  this  case  the  rater,  to  focus  on  the  repairs 
rather  than  the  message  itself. 
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46.  The  interpretation  of  the  speaker's  perspective  is  accurate  . NA  Y  N 

Errors 

46.A  Distance  of  the  speaker's  perspective  from  the  topic  or 

action  is  interpreted  inaccurately  (e.g.  people  standing  at  a 

distance  and  people  standing  near  by)  . NA  0  12  3  4 

46. B  The  focus  of  the  action  or  the  event  is  inaccurate  (e.g. 

discussing  the  knot  on  the  tree  trunk,  not  the  entire  tree) 
. NA  0  12  3  4 


47.  The  interpreter  repairs  errors  accurately  . NA  Y  N 

Errors 

47.A  Errors  are  unnoticed  by  interpreter  (i.e.  not  repaired)  . NA  0  12  3  4 

47. B  Errors  are  repaired  inaccurately  . NA  0  12  3  4 

47. C  Errors  are  repaired  awkwardly  (e.g.  rolling  the  eyes  in 

frustration)  . NA  0  12  3  4 

47. D  Errors  are  repaired  excessively  (e.g.  excessive  false 

starts)  . NA  0  12  3  4 
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48.  DEF:  If  the  interpretation  is  at  a  quick  pace  or  if  the  pace  quickens  during  a 
presentation  it  can  affect  the  interpretation.  The  meaning  of  speed  here  not  only  refers 
to  words  per  minute,  but  also  speed  of  the  delivery  related  to  the  nature  of  the 
information.  The  speaker  can  be  speaking  at  a  relatively  slow  pace  but  the  information 
is  'fast'  for  the  process  of  interpretation  (e.g.  technical  content  or  reading  a  prepared 
speech).  Due  to  fatigue,  both  physical  and  mental,  the  number  of  errors  can  increase  in 
longer  presentations.  The  number  of  errors  may  increase  with  a  change  in  pace  and  the 
interpretation  may  resemble  English  structure  which  are  both  errors. 


49.  DEF:  When  an  interpreter  is  unsure  of  the  utterance,  the  interpretation  should 
reflect  this  unsureness.  The  interpretation  should  not  be  seen  as  stated  definitely  by  the 
speaker.  There  should  be  a  clear  distinction  between  the  interpreter's  and  the  speaker's 
uncertainty.  For  example,  if  the  interpreter  is  unsure  if  the  speaker  said  Terry  or 
Gary,  then  this  should  be  made  clear  to  the  addressee  that  it  is  the  interpreter's 
uncertainty  and  not  the  speaker's. 
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48.  Maintains  the  interpretation  accuracy  regardless  of  the 

speaker's  pace  . NA  Y  N 

Errors 

48.A  Errors  increase  as  the  speaker's  pace  increases  . NA  0  12  3  4 

48. B  Errors  increase  as  the  speaker's  pace  slows  down  . NA  0  12  3  4 

48. C  More  errors  occur  at  the  end  of  the  interpretation  than  at 

the  beginning  . NA  0  12  3  4 

48. D  The  ASL  structure  changes  to  a  more  English  like 

interpretation  when  the  speaker's  pace  changes  . NA  0  12  3  4 


49.  Expresses  uncertainty  when  not  understanding  or  unsure  of  the 

utterance  appropriately  . NA  Y  N 

Errors 

49. A  When  the  interpreter  is  unsure  of  the  message,  there  is  no 

quizzical  expression  to  indicate  this  . NA  0  12  3  4 

49. B  Unclear  whether  it  is  the  interpreter's  or  speaker's 

uncertainty  . NA  0  12  3  4 
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50.  DEF:  ASL  requires  that  referents  be  clear  when  the  topic  of  the  discussion  or  the 
sentence  referred  to  originally  is  referred  back  to  a  second  time.  In  English,  as  in  other 
languages,  pronouns  are  used  to  refer  to  something  talked  about  earlier.  In  ASL  these 
must  be  interpreted  in  relation  to  the  message,  that  is,  the  components  must  be  linked 
together  in  order  to  be  consistent  with  the  source  language  message.  If  it  is  not  linked, 
the  interpretation  is  left  up  to  the  addressee  which  is  not  his/her  responsibility.  If  the 
message  is  linked  in  English  it  should  be  linked  in  ASL.  This  skill  area  does  not  mean  if 
the  English  is  unclear  that  the  ASL  interpretation  be  made  clear.  Unless  the  speaker  is 
redundant,  referring  back  to  the  topic  should  not  be  done  excessively. 


51.  DEF:  When  the  speaker  is  talking  about  himself  and  using  personal  pronouns  such 
as  T  and  'me',  the  interpretation  should  also  use  the  ASL  personal  pronoun  'ME'  and  not 
'S/HE-INDEX',  pointing  to  the  speaker.  A  clear  distinction  should  be  made  between  T  the 
speaker  and  T  the  interpreter.  Of  course,  at  times,  an  interpreter  will  refer  to  himself 
or  herself  when  speaking  directly  to  the  addressee  or  the  speaker.  This  should  be  clear 
and  there  should  be  an  observable  difference. 


Do  not  copy 


For  research  purposes  only  © 


M.  Taylor 


■i 


June  1993 


201 


50.  Refers  back  to  the  topic  when  a  pronoun  is  used  accurately  (e.g. 

The  stock  Aldus  is  doing  well.  It  (INDEX)  should  be  bought 

immediately.)  . NA  Y  N 

Errors 

50.A  Does  not  refer  back  to  the  topics  . NA  0  12  3  4 

50. B  Refers  back  to  the  topics  excessively  . NA  0  12  3  4 

50. C  Refers  back  to  the  inaccurate  topics  . NA  0  12  3  4 


51  .  Interprets  the  speaker's  comments  in  the  first  person  accurately  . NA  Y  N 

Errors 

51. A  Refers  to  speaker  as  INDEX-S/HE  +  SEE  when  the  speaker 

says  "I  saw..."  . NA  0  12  3  4 

51. B  Distinctions  between  i  the  speaker  and  1  the  interpreter 
are  not  accurate  (e.g.  the  interpreter  wants  to  tell  the 
addressee  that  she  is  having  difficulty  hearing  the  speaker) 

. NA  0  12  3  4 
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52.  DEF.  Eye  contact  with  the  addressee  or  the  camera  is  imperative.  One  uses  this  eye 
contact  to  determine  if  the  message  is  coming  across  to  the  deaf  person  by  observing  the 
addressees  facial  expression  such  as  nodding,  quizzical  looks  or  shaking  of  the  head 
indicating  s/he  is  not  understanding.  This  eye  contact  also  establishes  the  sightline  with 
which  space  is  used.  For  example,  signs  such  as  HE,  SHE,  and  OVER  THERE  are  all  off  the 
sightline,  whereas  ME  and  YOU  are  on  the  sightline. 


53.  DEF:  The  use  of  two  hands  in  ASL  is  imperative.  The  passive  hand  is  used  in  a 
variety  of  ways.  For  example,  the  passive  hand  can  be  used  to  indicate  a  chair  in  the 
corner  of  the  house  by  indexing  or  using  a  classifier,  while  the  dominant  hand  is  showing 
where  the  person  walked.  Hand  dominance  should  not  change  haphazardly  such  as  FOREST 
or  BLAME  signed  with  the  passive  hand  moving.  Dominance  should  remain  constant  and 
should  only  change  when  there  is  a  linguistic  reason  to  do  so. 
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52.  Eye  contact  is  appropriate  . IMA  Y  N 

Errors 

52.A  Eye  contact  with  the  addressee/camera  is  not  present  (e.g. 

looking  at  floor,  looking  at  speaker)  . NA  0  12  3  4 

52. B  Eye  contact  with  the  addressee/camera  is  excessive  (e.g. 

staring)  . NA  0  12  3  4 


53.  Accurately  uses  both  hands  for  signing  (e.g.  indexing  on 

nondominant  hand,  indicating  a  spatial  relationship)  . NA  Y  N 

Errors 

53. A  The  passive  hand  is  not  used  when  it  could  be  (i.e. 

underused)  . NA  0  12  3  4 

53. B  Changes  dominance  for  no  linguistic  reason  . NA  0  12  3  4 
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54.  DEF:  The  signing  in  the  interpretation  should  be  smooth  and  not  choppy  or  jerky. 
This  can  cause  inaccurate  linguistic  phrasing.  The  signs  should  flow  from  one  to  another. 
The  signs  should  not  be  signed  all  together,  that  is,  slurred.  The  interpretation  should  be 
confident  and  smooth,  moving  from  one  sign  to  the  next  with  ease  and  without  hesitation. 


55.  DEF:  Signs  should  be  made  in  a  comprehensible  manner.  If  the  signing  is  too  large 
(e.g.  the  arms  are  fully  extended  or  going  off  the  screen),  then  it  is  an  error.  If  the 
signing  is  so  small,  that  it  is  difficult  to  comprehend  the  interpretation,  it  is  an  error  as 
well.  The  signing  size  should  vary  according  to  the  setting.  If  it  is  a  one  on  one  doctor's 
appointment,  the  signing  would  be  smaller  than  if  interpreting  to  a  large  audience  of  50 
people.  If  there  is  a  change  in  size  of  signing  there  should  be  a  linguistic  reason.  It 
should  not  be  the  case  that  small  signing  all  of  a  sudden  has  2  or  3  large  signs  for  no 
linguistic  reason. 
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54.  The  signing  is  controlled  (e.g.  smooth)  . NA  Y  N 

Errors 

54. A  The  flow  is  choppy/jerky  (e.g.  start  and  stop  in 

linguistically  inaccurate  places,  stuttering  or  repetition  of 

signs  such  as  HAVE)  . NA  0  12  3  4 

54. B  The  flow  is  hesitant  (e.g.  the  interpreter  appears  to  need 

time  to  think  or  is  unsure  how  to  convey  the  message)  . NA  0  12  3  4 


55.  The  size  of  signing  is  appropriate  . NA  Y  N 

Errors 

55. A  The  movements  of  the  signs  are  too  large  (e.g.  causing 

distraction,  going  off  the  television  screen)  . NA  0  12  3  4 

55. B  The  movements  of  the  signs  are  too  cramped  (e.g. 

unreadable,  no  distinction  between  two  different  signs)  . NA  0  12  3  4 

55. C  Signing  size  varies  between  large  and  small  for  no 

linguistic  reason  . NA  0  12  3  4 
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COMPOSURE  AND  APPEARANCE 


Rating  Key 


=  Not  applicable,  not  in  the  target  language,  ASL 
Y  =  Yes,  the  skill  was  performed  accurately  100%  of 

the  time 

N  =  No,  the  skill  was  not  performed  accurately  100% 

of  the  time 


O 

1 

2 

3 

4 


Not  applicable,  not  in  the  target  language,  ASL 
No  errors  observed 


Low  frequency, 
High  frequency, 
Low  frequency, 
High  frequency  , 


low  severity 
low  severity 
high  severity 
high  severity 
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COMPOSURE  AND  APPEARANCE 

56.  DEF:  The  interpretation  should  convey  the  speaker's  meaning  and  intent  and  not  the 
reactions  or  feelings  of  the  interpreter.  Personal  interjections  should  not  be  observed 
in  the  interpretation.  The  interpreter's  opinions  should  not  show.  Only  the  speaker's 
ideas  and  opinions  should  be  conveyed.  The  interpreter  should  also  maintain  composure. 
For  example,  raising  hands  or  using  gestures  that  are  not  part  of  the  speaker's  message 
is  not  maintaining  composure.  The  intent  of  the  speaker  should  be  conveyed,  not  the 
intent  of  the  interpreter.  The  message  should  be  interpreted  prior  to  the  interpreter 
giving  any  reactions.  At  times,  it  is  appropriate  to  react  to  the  message  but  the  message, 
whether  it  is  a  joke  or  a  horror  story,  must  be  interpreted  prior  to  the  interpreter's 
reactions.  If  the  interpreter  laughs  prior  to  revealing  the  punch  line  of  a  joke,  it  is 
inaccurate.  Also,  when  everyone  else  is  clapping  enthusiastically,  it  is  culturally 
appropriate  for  the  interpreter  to  clap.  The  interpreter  should  respond  in  a  culturally 
appropriate  manner. 
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COMPOSURE  AND  APPEARANCE 

56.  The  interpreter  is  composed  while  interpreting  . NA  Y  N 

Errors 

56. A  Reacts  to  source  language  message  prior  to  interpreting  it 

(e.g.  laughs  at  joke  prior  to  interpreting  the  message)  . NA  0  12  3  4 

56. B  Displays  the  interpreter's  own  emotions  inappropriately 

(e.g.  puzzlement,  confusion,  worry)  . NA  0  12  3  4 

56. C  Reacts  to  his/her  own  errors  (adding  apologies  for  the 

misinterpretations  for  example,  EXCUSE  ME,  SORRY)  . NA  0  12  3  4 

56. D  Does  not  react  when  culturally  appropriate  (e.g.  All  are 
laughing  but  the  interpreter  remains  neutral  with  no 

response)  . NA  0  12  3  4 
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57.  DEF:  There  should  be  control  of  personal  mannerisms.  If  the  body  is  tense  then  the 
shoulders  might  inch  upward.  Or  if  the  body  is  too  relaxed,  the  torso  might  slouch  or 
become  overly  casual.  When  there  is  a  pause  in  the  interpretation,  the  hands  should  be 
comfortably  placed  in  the  lap  or  if  standing,  down  by  the  sides.  Clasping  the  hands 
together  in  front  of  the  waist  may  be  permissible.  However,  due  to  the  large  number  of 
repetitive  motion  injuries  (e.g.  tendonitis,  carpal  tunnel  syndrome)  associated  with  sign 
language  interpreting,  it  is  healthier  to  have  the  hands  in  a  relaxed  position  on  the  lap  or 
by  the  sides  rather  than  clasped  at  waist  level  or  above  when  the  blood  can  not  easily 
flow  to  the  finger  tips. 
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57.  Good  posture  is  maintained  . NA  Y  N 

Errors 

57.A  The  head  moves  down  to  meet  the  signs  (e.g.  FATHER)  . NA  0  12  3  4 

57. B  Slouching  is  present  . NA  0  12  3  4 

57. C  Shoulders  are  raised  . NA  0  12  3  4 

57. D  The  body  posture  is  distracting  (e.g.  swaying,  walking 

around,  legs  swinging,  one  hip  thrust  outward)  . NA  0  12  3  4 

57. E  Hands  remain  at  chest  or  waist  level  during  breaks  in  the 
utterance  instead  of  placing  hands  down  in  lap  or  at  the 
sides  . NA  0  12  3  4 
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58.  DEF:  The  interpreter  should  be  calm  and  relaxed,  keeping  all  distracting 
mannerisms  from  view.  The  interpretation  should  flow  without  the  face  reflecting 
internal  mental  processing.  Behaviors  such  as  pushing  up  one's  glasses,  or  playing  with 
facial  hair  such  as  a  mustache  should  not  be  seen.  Facial  expressions  that  distract  the 
addressee,  for  example,  excessive  frowning  or  twitching  of  any  kind  should  not  be 
observable. 


59.  DEF:  The  interpreter's  attire  can  affect  the  interpretation.  The  clothes  should 
contrast  in  color  with  the  pigment  of  the  skin  and  should  not  have  shiny  or  flowery 
patterns.  If  jewelry  is  worn  it  should  not  be  distracting.  The  hair  should  be  neatly 
arranged  and  should  not  fall  into  the  eyes,  or  if  long,  get  tangled  with  the  hands. 
Mustaches  and  beards  should  be  neatly  trimmed. 
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58.  The  interpreter's  mannerisms  are  appropriate  . NA  Y  N 

Errors 

58. A  The  addressee  can  perceive  the  internal  thinking  process 

on  the  interpreter's  face  . NA  0  12  3  4 

58. B  The  head  is  distracting  (e.g.  bobbing)  . NA  0  12  3  4 

58. C  The  hands  are  distracting  (e.g.  pushing  up  glasses,  playing 

with  hair)  . NA  0  12  3  4 

58. D  Gestures  are  distracting  (e.g.  makes  a  mistake  and  raises 

hands  in  exasperation)  . NA  0  12  3  4 

58. E  Facial  expressions  are  distracting  (e.g.  yawning, 

twitching,  grimacing)  . NA  0  12  3  4 


59.  The  interpreter's  appearance  is  appropriate  . NA  Y  N 

Errors 

59. A  Jewelry  is  distracting  (e.g.  earrings,  watch)  . NA  0  12  3  4 

59. B  Clothing  is  distracting  (e.g.  sparkling  or  flowery  shirt, 

floppy  sleeves)  . NA  0  12  3  4 

59. C  The  hair  (facial  hair  included)  is  distracting  . NA  0  12  3  4 

59. D  Make-up  is  distracting  (e.g.  nail  polish,  lipstick,  eye 

shadow)  . NA  0  12  3  4 


Do  not  copy 


For  research  purposes  only  © 


M.  Taylor 


June  1993 


213 


REMINDERS 


-Be  sure  to  refer  to  the  work  or  the  interpretation,  not  the 

interpreter,  when  writing  comments.  Avoid  using  'you',  for 
example,  "you  signed..."  or  "your  interpretation..."  Instead  use 
"the  interpretation  demonstrated  signs  that..."  or  the 
"interpretation  was..." 

-Be  sure  to  write  both  positive  and  negative  comments  when 
providing  feedback  about  the  interpretation. 

-View  the  tape  at  least  twice  without  sound. 

-Be  sure  to  rate  every  skill  and  every  error. 

-If  the  skill  is  not  present,  then  it  is  marked  NA,  not  applicable,  and 
all  of  the  errors  related  to  this  skill  are  also  marked  NA. 

-Remember  to  rate  the  errors  associated  with  the  skill  it  defines 

only.  The  errors  should  not  be  rated  as  if  they  were  associated 
with  other  skills  For  example,  the  error  5.B  "rhythm  or 
cadence  is  inaccurate"  should  be  rated  in  association  with 
skill  5.  "Fingerspelling  is  used  for  emphasis  accurately".  It 
should  not  be  rated  for  overall  rhythm  or  cadence  of  the 
fingerspelling  in  the  interpretation. 
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RATER'S  IMPRESSIONS 
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RATER'S  IMPRESSIONS 

This  segment  is  reserved  for  comments  associated  with  the  interpretation  task.  No 
rating  per  se  will  occur.  Impressions  will  be  documented  here  in  the  rater's  own  words 
in  narrative  form. 

60.  Is  the  conduct  the  interpreter  exhibits  acceptable  (e.g.  clearly  identifys  if  it's 
the  interpreter's  or  the  speaker's  mistake,  culturally  appropriate  manner  of 
interrupting  the  speaker,  not  using  personal  interjections  such  as  LECTURE  + 
BORING)?  Why  or  why  not?  Provide  several  examples. 


61.  Does  the  target  message  reflect  the  source  message?  Why  or  why  not?  Provide 
several  examples. 
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62.  Is  the  ASL  semantic  base  sufficient  for  the  interpreting  task?  Why  or  why  not? 
Provide  several  examples. 


63.  Was  the  target  language  ASL?  Why  or  why  not?  Provide  several  examples. 
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64.  Does  the  interpreter's  own  values,  beliefs,  or  biases  skew  the  interpretation  (e.g. 
interpreting  the  word  'chairman'  as  CHAIRWOMAN  or  interpreting  The  Great  One  as 
JESUS  when  interpreting  for  Natives  of  North  America)?  Provide  several  examples. 


65.  Throughout  the  interpretation,  how  well  is  the  same  interpreting  quality 
maintained  from  beginning  to  end?  Provide  several  examples. 
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66.  Other  comments  the  rater  would  like  to  make. 


“Document  time  and  date  completed:. 
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CROSS  REFERENCE  INDEX 
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TOPIC _ ITEM  NIJMBFR 

Acronyms . 2 

Additions . 42.  A 

Affect . 44. A,  44. B,  44. C 

Age  appropriate . 21  .A 

Agreeement . 29. B 

Appearance 


(see  composure) 
Characterization 


perspective  taking . 30 

Clarity . 7,  15,  19.  E 

Classifiers  and  SASSes . 22-28 

Composure  and 

appearance . 56-59 

Compound  signs . 18. C 

Conduct . 6  0 

Cultural  adjustments . 42. C,  56. D 

Deletions 

(see  omissions) 

Distractions . 58,  59 

Double-handed  . 

sians . 19. H,  24. D,  26. D 
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TOPIC . ITEM  NUMBER 

Emphasis . 5 

English 

comprehension . 6 1 

endings . 18. G 

initialized  signs . 18. F,  432. B 

mouthing . 43. D 

sign  systems . 20. B 

structure . 43. A,  48. D 

Error  repair . 4  7 

Eye  gaze . 5. A,  38. C,  52 

Fingerspelling . 1  - 1  0 

spelling . 1  -A 

First  person . 51 

Geography . 29. D 

Grammar . 36-41 

Handshapes . 19. A,  22. A,  23. A 

Incorporation 

numbers . 1  8 

Indexing . 35 

Intent . 44 

Interpreting . 42-55,  63 

Loan  signs . D 
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TOPIC . ITEM  NUMRFR 

Location  of 

numbers . 12. D 

signs . 19. C 

Mistakes 

(see  error  repair) 

Monitoring 

(see  self-monitoring) 


Mouthing . 1  0 

Movement . 8,  12.C,  15. B,  19.B,  19. F,  39. A,  39. B 


Nondominant  hand 
(see  passive  hand) 

Nonmanual  facial 


grammar . 18. B,  19. G,  22. B,  23. B,  33. D,  36. A, 

37. A, 38,  . 40. B,  41. B,  43. E 


Noun-verb  pairs . 18.E 

Numbers . 1  1-17 

(see  incorporation) 

numbering  systems  . 12 

number  signs 

non-specific . 1  4 

Omissions . 3,  1 1.B,  42. B 


Pace . 

speaker's  pace 

Palm  orientation... 

Passive  hand . 


9,  16,  17 
48, 

12.B.1 9.D 
32. B,  53 


Do  not  copy 


For  research  purposes  only  © 


M.  Taylor 


June  1993 


223 


TOPIC . ITEM  ni  imrfr 

Pausin9 . 36. B,  37. B,  43. C 

Pictorial  outlining . 26 

Plurals . 39 

Pronominalization . 34 

Posture . 5  7 

Questions . 37 

Reality  principle . 29. C 

Redundancy . 5.C,  20. C,  30. D,  35. C,  36. D,  37. D 

Register . 45 

Rhythm . 5.B,  6 

SASSes 


(see  classifiers  and  SASSes) 

Semantics . 18. A,  62 

Space 

(see  structruing  space) 

Speed 

(see  pace) 

Spelling 

(see  fingerspelling) 


Statements . 3  6 

Structuring  space . 2  9-35 

restructuring . 31  .B 
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TOPIC 


ITEM  NUMBER 


Temporal  aspect . 4  1 

Time  . 40 

Verts . 24. B,  33 

body-anchored . 33. C 

directional  . 33. B 

Visual  accuracy . 2  4 

Vocabulary . 1  8-21 

Diversity . 2  0 

Regional . 21  .B 
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RATER'S  INSTRUCTION  MANUAL  for 

TAYLOR'S  DIAGNOSTIC  INSTRUMENT: 

English  to  American  Sign  Language 
Interpretation  Assessment 
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PURPOSE  AND  GOALS 

Taylor's  Diagnostic  Instrument  is  a  self-instructive  diagnostic 
instrument  for  interpreter  educators  working  in  the  field  of 
American  Sign  Language/English  and  who  evaluate  interpretations  of 
interpreters'. 

The  purpose  of  this  instrument  is  to  diagnostically  assess 
video  taped  interpretations  from  English  to  ASL.  This  instrument 
assesses  neither  ASL  to  English,  nor  transliteration  skills. 

The  instrument  assesses  a  particular  interpretation  an 
interpreter  or  student  interpreter  provides.  It  is  a  specific 
assessment  of  a  specific  piece  of  work.  It  measures  how  well  the 
message  of  the  interpretation  matches  the  source  language,  English. 

The  goal  of  the  instrument  is  to  provide  a  standardized  format 
for  viewing  and  assessing  ASL  interpretations.  By  providing 
standardized  diagnoses,  interpreters  will  have  an  opportunity  to 
develop  their  skills  based  on  the  specific  criteria  contained  in  this 
diagnostic  instrument  and  the  degree  to  which  they  achieved  these 
skills. 
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BRIEF  DESCRIPTION 


This  instrument  provides  a  standardized  format  to  view 
English  to  ASL  video  taped  interpretations.  It  organizes  large 
numbers  of  skills  according  to  Major  Features. 

Eight  Major  Features  are  presented  in  the  following  order: 

1)  Fingerspelling 

2)  Numbers 

3)  Vocabulary 

4)  Classifiers/SASSes  (Size  and  Shape  Specifiers) 

5)  Structuring  Space 

6)  Grammar 

7)  Interpreting 

8)  Composure  and  Appearance 

Within  the  Major  Features  a  common  format  is  used.  Each 
listed  skill  is  followed  by  a  related  list  of  possible  errors  along 
with  a  corresponding  definition.  In  addition,  the  Major  Features  are 
followed  by  an  overall  narrative  assessment,  Rater's  Impressions. 


TERMINOLOGY 


The  two  languages  used  in  this  instrument  are  English  and 
American  Sign  Language  (ASL).  English  is  the  source  language,  the 
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language  first  uttered,  and  ASL  is  the  target  language,  the  language 
into  which  the  interpretation  is  made. 

There  are  at  least  two  consumers,  one  hearing  and  one  Deaf, 
referred  to  in  the  instrument.  The  hearing  consumer,  speaking 
English,  is  referred  to  as  the  speaker.  It  is  this  person  who  initiates 
the  source  language  sample,  English.  The  Deaf  consumer  is  the 
person  to  whom  the  interpretation  is  provided. 

Persons  using  this  assessment  instrument  are  raters  and  the 
persons  undergoing  the  assessment  are  interpreters. 

The  dominant  hand  is  the  active  hand.  The  non-dominant  hand 
is  the  passive  or  base  hand.  For  right-handed  signers  these  are  the 
right  and  the  left  hand,  respectively. 

In  the  text  of  the  manual  and  instrument,  ASL  is  written  in 
capital  letters.  Hyphens  are  used  between  words  or  between 
individual  letters  to  indicate  that  one  sign  is  used  or  that  the  word 
is  fingerspelled.  For  example,  SHAVE-ON-HEAD  indicates  one  sign, 
whereas,  B-A-R-B-A-R-A  indicates  a  fingerspelled  word.  If  more 
than  one  sign  is  used  then  plus  symbols  are  written  between  each 
different  sign.  For  example,  MOM  +  GO  +  HOME,  indicates  three 
separate  signs. 
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ASSUMPTIONS  AND  PRIOR  CONDITIONS 

The  basic  assumptions  and  prior  conditions  are: 

-The  rater  is  qualified  and  has  the  knowledge  and  ability  to  assess 
the  interpretation  accurately. 

-The  rater  must  have  access  to  both  the  English  message  and  ASL 

interpretation  to  evaluate  the  work,  especially  to  evaluate  the 
message  equivalency. 

-The  text,  the  amount  of  preparation,  prior  knowledge  of  the 

particular  stimulus,  and  the  amount  of  related  experience  the 
interpreter  has,  will  all  affect  the  accuracy  of  the 
interpretation. 

-Accurate  and  appropriate  skills,  and  errors,  can  be  observed  within 
the  same  sample  of  work. 

-When  the  interpretation  does  not  match  the  source  message,  errors 
are  present. 

-Errors  skew  the  speaker's  message. 


LIMITATIONS 

This  instrument  diagnostically  assesses  interpretations  from 
English  to  ASL.  It  does  not  apply  to  transliteration,  and  therefore, 
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should  never  be  used  to  assess  transliterating  skills  or  other  sign 
systems  used  to  represent  English. 

Although  this  instrument  assesses  language,  it  only  assesses 
it  through  the  act  of  interpreting.  It  has  not  been  developed  for 
American  Sign  Language  assessments  and  should  not  be  used  prior  to 
interpreting  education. 

The  instrument  is  manageable,  but  it  is  not  exhaustive.  The 
skills  and  possible  errors  included  in  the  instrument  represent  the 
key  requirements  for  an  effective  interpretation  as  validated  by  a 
national  panel  review  and  two  field  studies  involving  experts  from 
Canada  and  the  United  States. 

Due  to  the  specific  nature  of  the  instrument  and  the  need  to 
provide  the  interpreter  with  a  sample  of  his/her  work  with  the 
diagnostic  assessment,  the  interpretation  must  be  on  video  tape.  It 
is  not  designed  to  assess  live  interpretation  performances. 

Depending  on  the  rater's  background  and  ability  to  detect 
errors,  there  may  be  false  positives  (i.e.  an  error  occurs  in  the 
interpretation,  but  the  rater  overlooks  the  error  and  marks  it 
accurate  instead  of  inaccurate).  The  rater  must  have  a  keen  eye  for 
detecting  errors.  Without  this  perspective  and  awareness,  an  error 
may  go  unnoticed  and  marked  inaccurately. 
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RATER  QUALIFICATIONS 

People  using  this  instrument,  raters,  will  be  interpreter 
educators  with  a  minimum  of  two  years  experience  evaluating 
interpreters  either  on  the  job  (e.g.  practicum  supervisors  or  lead 
interpreters)  or  teaching  interpretation  skills  to  students. 
Experience  teaching  professionalism  or  other  theory-based  courses 
is  not  sufficient  background  to  use  this  instrument.  Educators  must 
be  well-versed  in  ASL  grammar  and  semantics,  as  well  as 
interpretation  processes. 

The  field  is  rapidly  changing.  Therefore,  educators  must  be 
active  interpreters  and  members  of  the  national  interpreter's 
organization,  that  is,  either  the  Association  of  Visual  Language 
Interpreters  of  Canada  (AVLIC)  or  the  Registry  of  Interpreters  for 
the  Deaf  (RID).  In  addition,  educators  must  be  active  members  of  the 
Canadian  Association  for  the  Education  of  Sign  Language 
Interpreters  (CAESLI)  in  Canada  or  the  Conference  of  Interpreter 
Trainers  (CIT)  in  the  United  States.  Educators  must  hold  national 
certification,  either  the  Certificate  of  Interpretation  offered  by 
AVLIC  in  Canada  or  the  Certificate  of  Interpretation  or  the 
Comprehensive  Skills  Certificate  offered  by  RID  in  the  United 

States. 
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INTERPRETER  QUALIFICATIONS 

Interpreters  undergoing  the  assessment  must  be  either 
interpreting  students  who  have  mastered  at  least  some  interpreting 
skills  or  interpreters  with  experience  in  the  field  of  English/ASL 
interpretation.  Students  who  have  learned  ASL  only  and  have  not 
learned  interpreting  are  not  to  be  assessed  using  this  instrument. 
The  interpreters  must  have  at  least  minimal  competencies  through 
direct  training  and/or  work  experience  to  be  assessed  using  this 
diagnostic  instrument. 


INSTRUMENT  ORGANIZATION 


The  eight  Major  Features  are  sequenced  in  the  following  order: 
Fingerspelling,  Numbers,  Vocabulary,  Classifiers  and  Size  and  Shape 
Specifiers,  Structuring  Space,  Grammar,  Interpretation,  and 
Composure  and  Appearance.  The  sequence  is  designed  to  move  the 
rater  through  the  categories  with  ease.  As  determined  by  a  panel  of 
experts,  each  Major  Feature  includes  a  list  of  skills,  sequenced  from 
most  essential  to  least  essential  followed  by  a  list  of  possible 
errors  related  to  each  skill.  All  of  the  skills  and  errors  are 
numbered  consecutively  from  the  beginning  to  the  end  of  the 
instrument;  they  are  not  numbered  by  feature.  Following  each  list  of 
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possible  errors  is  a  series  of  blank  lines  for  writing  comments,  and 
providing  feedback  and  examples  to  help  clarify  the  meaning  of  the 

rating  the  rater  gives  the  interpretation.  An  example  is  provided  in 
Figure  1. 

Figure  1:  Example  of  skill  #47  with  related  errors  from  the 
_ Major  Feature  of  Interpreting. _ _ _ 


47.  The  interpreter  repairs  errors  accurately  . NA  Y  N 

Errors 

47. A  Errors  are  unnoticed  by  interpreter  (i.e.  not 

repaired)  . NA  0  1  2  3  4 

47. B  Errors  are  repaired  inaccurately  . NA  0  12  3  4 

47. C  Errors  are  repaired  awkwardly  (e.g.  rolling  the 

eyes  in  frustration)  . NA  0  12  3  4 

47. D  Errors  are  repaired  excessively  (e.g.  excessive 

false  starts)  . NA  0  12  3  4 


Corresponding  to  each  skill,  on  the  opposite  page  from  the 
skill,  are  definitions.  An  example  is  provided  in  Figure  2.  The 
definitions  accompanying  each  skill  are  intended  for  general,  not 
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specific,  applications.  There  are  a  multitude  of  exceptions  and  not 
all  of  them  are  included  in  this  instrument.  For  example,  it  is 
generally  accepted  that  the  number  of  false  starts  should  not  be 
excessive.  However,  if  this  accurately  reflects  the  speaker's 
message;  that  is,  if  the  speaker  has  an  excessive  number  of  false 
starts,  then  the  interpretations  of  false  starts  are  appropriate  and 
are  not  considered  errors. 


Figure  2:  Definition  #47  corresponding  to  skill  #47  within  the 
_ Major  Feature  of  Interpretation _ 

47.  DEF:  It  is  natural  for  errors  to  occur  while  interpreting.  Once 
errors  are  identified  they  should  be  repaired  accurately  and 
repaired  with  ease.  If  they  are  repaired  with  apologies  (e.g.  SORRY 
or  EXCUSE  ME)  this  would  be  considered  an  awkward  repair.  If 
information  is  missed  or  not  understood,  this  should  be  reflected  in 
the  interpretation  and  is  not  considered  an  error  within  this 
category.  Excessive  repairing  should  be  avoided.  If  errors  are 
repaired  excessively  it  means  that  a  distraction  occurs  causing  the 
viewer,  in  this  case  the  rater,  to  focus  on  the  repairs  rather  than 
the  message  itself. 


The  final  section  of  the  instrument  to  be  completed  is  the 
Rater's  Impressions  which  gives  the  rater  a  place  to  write  general 
comments  and  impressions.  This  is  formatted  slightly  differently 
from  the  rest  of  the  instrument.  It  is  constructed  with  questions 
the  rater  answers  in  narrative  form.  No  lists  of  skills  or  possible 
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errors  are  present.  It  is  simply  a  place  for  the  rater  to  write 
comments  that  are  not  addressed  earlier  or  that  the  rater  wants  to 
make  explicit  in  the  feedback  to  the  interpreter.  Figure  3  provides 
an  example. 


Figure  3:  Example  from  Rater's  Impressions:  #64 _ 

64.  Does  the  interpreter's  own  values,  beliefs,  or  biases  skew  the 
interpretation  (e.g.  interpreting  the  word  'chairman1  as 
CFIAIRWOMAN  or  interpreting  The  Great  One  as  JESUS  when 
interpreting  for  Natives  of  North  America)?  Provide  several 
examples. 


Overlap  occurs  throughout  the  instrument.  Several  skills,  such 
as  nonmanual  facial  grammar,  occur  in  more  than  one  skill  area. 
Overlap  allows  cross-referencing  of  skills  across  Major  Features. 

At  the  back  of  the  instrument,  the  Cross  Reference  Index  lists 
where  a  particular  skill  appears  in  the  instrument.  This  reference 
section  addresses  the  possibility  that  a  skill  or  error  may  occur  in 
more  than  one  location.  It  can  be  used  as  a  cross  reference  to  assist 
in  rating  the  interpretation. 
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USING  AND  DEVELOPING  STIMULI 

The  instrument  can  be  used  with  a  variety  of  English  source 
language  texts.  One  five-minute  audio  tape  accompanied  with  a 
written  transcript  is  provided  with  the  instrument. 

If  the  rater  chooses  to  develop  materials,  the  English  source 
language  tape  should  include  texts  that  will  allow  the  interpreter  an 
opportunity  to  demonstrate  skills  from  all  of  the  Major  Features, 
particularly  the  following: 

-  Fingerspelling 

-  Numbers 

-  Classifiers  and  Size  and  Shape  Specifiers 

-  Structuring  Space 

The  remaining  four  Major  Features  will  automatically  arise  in  any 
English  source  language  text. 

-  Vocabulary 

-  Grammar 

-  Interpreting 

-  Composure  and  Appearance 

The  equipment  used  for  video  taping  the  interpretation  sample 
should  be  of  excellent  quality  and  able  to  pick  up  both  the  visual  and 
auditory  information  necessary  for  rating  the  interpretation.  The 
camera  should  have  a  tight  focus  on  the  interpreter,  capturing  the 
top  of  the  head  and  the  torso.  There  may  be  deviations  from  this,  for 
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example,  the  interpreter  is  particularly  interested  in  distracting 
mannerisms  such  as  extraneous  body  movements  below  the  waist 
(e.g.  legs  shaking).  Otherwise,  there  should  be  no  need  to  include  the 
entire  body  on  the  screen. 


ADMINISTRATION  PROCEDURES 

Amount  of  Time  Required  to  Rate  Interpretations 

It  will  be  necessary  to  spend  about  1  to  1.5  hours  to  become 
familiar  with  the  instrument  and  approximately  2  hours  to  mark  a 
five  minute  interpretation  sample. 

First  time  users  should  carefully  read  the  manual  and 
instrument  prior  to  using  it.  In  this  way,  the  rater  will  become 
familiar  with  it,  thereby  allowing  more  time  for  the  actual  rating  of 
the  interpretation  rather  than  spending  time  understanding  how  to 
use  the  instrument.  Once  the  rater  is  familiar  with  the  instrument, 
the  manual  can  be  used  for  reference.  Other  than  to  refresh  the 
memory,  there  is  no  need  to  read  the  manual  every  time  an 
interpretation  sample  is  rated. 
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Requirements  Prior  to  Rating 

The  assessment  is  conducted  while  viewing  a  video  taped 
sample  of  interpretation.  The  television  monitor  screen  should  be  at 
least  19"  in  diameter.  Anything  smaller  reduces  the  visibility  of  the 
interpretation  and  may  skew  the  reliability  of  the  assessment. 

The  rating  is  based  on  the  knowledge  the  rater  has  about  the 
Deaf  consumer  and  the  setting.  The  rater  should  know  pertinent 
information  about  the  Deaf  consumer  such  as  the  preferred  signing 
style,  age,  and  the  region  he/she  is  from.  The  rater  should  also  know 
the  setting  where  the  interpretation  occurs,  for  example,  whether  it 
is  a  formal  or  informal  lecture  provided  to  a  large  audience  or  it  is 
an  intimate  one  to  one  session  with  a  counselor. 

Viewing  the  Video  Tape 

First  the  rater  should  preview  the  video  tape  from  beginning  to 
end.  He/she  should  check  to  be  sure  both  the  English  stimulus  and 
ASL  interpretation  have  been  recorded.  Interference  from  the  source 
language,  English,  may  cause  rating  errors  if  sound  is  heard 
simultaneously  while  analyzing  the  ASL  interpretation.  Therefore, 
the  rater  should  view  the  entire  interpretation  without  sound  at 
least  twice  during  the  assessment  procedures. 
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The  video  tape  must  be  reviewed  several  times  to  obtain  an 
accurate  assessment  of  the  interpretation.  The  sample  of  work  may 
be  viewed  as  many  times  as  necessary.  The  number  of  times  a  rater 
views  the  selected  tape  depends  on  the  rater's  experience  with  the 
instrument  and  stimulus.  The  rating  will  be  as  accurate  and 
informative  as  the  rater's  ability  to  notice  errors  and  to  reflect 
these  back  to  the  interpreter.  As  stated  earlier  the  time  required  to 
mark  a  five  minute  sample  is  approximately  2  hours. 

When  viewing  the  tape,  the  rater  should  try  to  avoid  looking  at 
the  same  material  repeatedly.  That  is,  assess  the  entire  sample,  not 
just  one  segment  of  the  work.  Raters  may  prefer  to  rewind  at 
certain  sections  rather  than  watch  the  entire  text  repeatedly  from 
beginning  to  end.  This  is  acceptable  as  long  as  the  entire 
interpretation  is  evaluated  and  certain  sections  are  not  viewed  more 
frequently  than  others. 


RATING  PROCESS 

Because  the  sequence  of  the  Major  Features  is  specifically 
designed  to  assist  the  rater  in  moving  easily  from  one  section  to  the 
next,  it  is  easier  to  follow  the  sequence  of  the  instrument  when 
rating  an  interpretation  than  to  move  haphazardly  from  one  section 
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to  another.  However,  at  times  a  rater  may  wish  to  make  a  comment 
but  is  not  sure  where  it  best  fits.  At  these  times  it  may  be  helpful 
to  use  a  blank  piece  of  paper  for  notes;  when  the  appropriate  section 
is  reached  use  the  notes  to  complete  the  rating.  If  it  is  more 
convenient,  the  Cross  Reference  Index  at  the  end  can  be  used  as  well 
to  locate  specific  skills  where  a  comment  might  best  fit. 

Two  parts  of  the  interpretation  are  rated.  The  first  is  the 
skills  themselves.  The  second  is  the  possible  errors  associated 
with  each  skill.  An  example  is  provided  in  Figure  1. 

Each  skill  is  followed  by  three  possible  rating  options: 

NA  =  Not  applicable,  not  in  the  target  language,  ASL 
Y  =  Yes,  the  skill  was  performed  accurately  100%  of  the  time 

N  =  No,  the  skill  was  not  performed  accurately  100%  of  the  time 

Each  possible  error  is  followed  by  five  possible  rating  options: 

NA  =  Not  applicable,  not  in  the  target  language,  ASL 
0  =  No  errors  observed 

low  severity 
low  severity 
high  severity 
high  severity 


1  =  Low  frequency, 

2  =  High  frequency, 

=  Low  frequency, 

=  High  frequency, 


3 

4 


Do  not  copy 


For  research  purposes  only  © 


M.  Taylor 


June  1993 


242 


The  rating  of  0-4  for  each  error  is  based  on  both  the  frequency 
and  the  severity  of  the  error. 

Frequency  refers  to  the  number  of  times  an  error  occurs  within 
the  interpretation.  It  is  either  rated  low  or  high  in  frequency.  'Low' 
means  it  occurs  seldom  or  occasionally,  once  or  twice  among  several 
opportunities.  'High'  in  frequency  means  it  occurs  repeatedly  or 
often.  Frequency  can  differ,  in  that  the  same  error  occurs  across 
items  or  the  same  error  occurs  often;  either  way  it  is  occurring 
repeatedly.  An  example  of  errors  occurring  across  items  is 
misspelling  different  words  in  several  parts  of  the  interpretation. 

An  example  of  an  error  occurring  repeatedly  is  Nazi  misspelled  as  N- 
A-T-Z-l  over  and  over  again. 

Severity  refers  to  inaccuracy  and  the  potential  for 
misunderstanding.  If  it  is  an  error,  it  is  inaccurate.  If  this  error 
does  not  skew  the  meaning,  it  is  considered  low  in  severity.  If  the 
error  skews  the  meaning  and  the  message  is  irretrievable  or  the 
meaning  greatly  altered,  then  it  is  high  in  severity.  Severity  is  more 
serious  than  frequency.  If  the  error  is  high  in  severity,  the  rating  is 
either  a  3  or  4,  depending  on  the  frequency. 


Examples  of  rating  errors  associated  with  skill  #47: 
#47. A  Errors  are  unnoticed  by  interpreter  (i.e.  not 

repaired) . 
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NA  =  It  was  not  applicable  because  no  interpretation  errors 
occurred.  Therefore,  there  was  no  need  to  repair  errors. 

0  =  Any  time  an  error  occurred  the  interpreter  at  least  attempted  to 
repair  it.  The  errors  never  went  unnoticed. 

1  =  A  few  interpretation  mistakes  were  unnoticed  (e.g.  Signs  which 

tend  to  be  two  handed  signs  in  a  formal  situation,  such  as 
HAVE,  were  signed  with  only  one  hand,  occasionally);  the 
frequency  was  low,  and  the  severity  was  low  because  the 
meaning  was  not  skewed. 

2  =  Many  interpretation  mistakes  were  unnoticed,  but  did  not  skew 

the  meaning  (e.g.  Signs  which  tend  to  be  two  handed  signs  in  a 
formal  situation,  such  as  HAVE,  were  signed  with  only  one 
hand,  consistently);  the  frequency  was  high  because  it 
occurred  often,  and  the  severity  was  low  because  the  meaning 
was  not  skewed. 

3  =  A  few  interpretation  mistakes  were  unnoticed  (e.g.  inaccurate 

numbers  were  used  occasionally);  the  frequency  was  low,  but 
the  severity  was  high  because  the  meaning  was  skewed. 

4  =  Many  interpretation  mistakes  were  unnoticed  (e.g.  names  or 

dates  were  often  inaccurate);  the  frequency  was  high  because 
it  occurred  repeatedly,  and  the  severity  was  high  because  the 
meaning  was  skewed. 
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If  the  skill  is  marked  NA,  not  applicable,  then  all  possible 
errors  for  that  specific  skill,  are  also  marked  NA.  Meaning,  if  the 
skill  was  not  demonstrated  then  all  related  errors  were  also  not 
demonstrated  and  therefore  NA,  not  applicable,  is  the  correct  rating 
option. 

If  the  skill  is  marked  Y  for  yes,  the  skill  was  performed 
accurately  100%  of  the  time,  then  all  possible  errors  for  that 
specific  skill,  are  either  NA  or  O.  For  example,  if  the  interpretation 
accurately  incorporated  numbers,  the  skill  would  be  marked  Y  for 
yes.  The  possible  errors  associated  with  this  skill  can  be  either  NA 
or  O.  If  there  was  no  use  of  number  incorporation  related  to  money 
then  the  rating  is  NA,  because  it  is  not  applicable;  it  is  not  in  the 
target  language.  However,  if  it  was  in  the  target  language  then  it  is 
marked  0,  no  errors  observed. 

If  the  skill  is  marked  N  for  no,  the  skill  was  not  performed 
accurately  100%  of  the  time,  the  rater  determines  the  error  or 
errors  present  and  scores  them  accordingly.  The  rating  for  the 
errors  may  differ  from  one  another.  For  example,  47. A=  NA;  47. B  = 
2;  47. C  =  0;  47. D  =  4. 

The  rater  should  not  choose  a  point  between  those  provided. 
For  example,  if  the  rater  feels  the  error  is  between  3  and  4,  the 
rater  must  choose  either  3  or  4  and  not  circle  both  or  indicate  a 
midpoint  like  3.5.  As  a  general  rule,  if  the  rater  feels  the  error  is 
between  two  certain  points  on  the  continuum,  the  rater  should 
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choose  the  higher  number,  that  is,  the  worse  error.  It  should  be  very 
clear  to  both  the  rater  and  interpreter  that  the  scores  marked  are 
for  providing  insightful  feedback  to  the  applicant.  A  consistent 
rating  pattern,  between  raters  and  within  the  same  rater  when 
rating  different  interpretations,  will  occur  if  the  rater  chooses  the 
higher  number  when  in  doubt. 

When  strengths  and  weaknesses  are  noticed,  these  should  be 
documented  on  the  blank  lines  provided  after  each  skill  and  list  of 
possible  errors.  A  few  comments  are  ample  to  illustrate  strengths 
or  areas  in  need  of  improvement.  The  rater  should  be  certain  to 
document  both  the  positive  and  the  negative  comments,  not  just 
negative.  The  rater  should  also  use  the  blank  lines  to  link  skills  that 
occur  across  different  areas.  For  example,  mentioning  a  common 
thread  in  the  interpretation  such  as  deleting  information,  would  be 
useful  to  the  interpreter. 

The  comments  written  in  the  instrument  will  be  used  as  one 
part  of  the  final  assessment  of  the  interpretation.  The  rater  must 
keep  in  mind  the  clarity  of  the  comments  so  when  the  interpreter 
receives  the  instrument,  the  comments  make  clear  the  strengths  and 
weaknesses  observed  in  the  interpretation. 
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RESULTS 


Once  the  interpreter  provides  an  interpretation  sample,  no 
more  than  one  month  should  pass  before  the  diagnostic  assessment 
is  returned  to  the  interpreter.  Ideally,  the  feedback  should  be 
returned  within  a  week  of  when  the  rater  receives  the  sample  of 
work.  The  completed  diagnostic  assessment  instrument  should  be 
given  to  the  interpreter  along  with  his/her  video  taped 
interpretation  sample.  The  rater  should  be  available,  either  by  phone 
or  in  person,  to  answer  questions  the  interpreter  might  have  about 
the  diagnostic  assessment. 

Since  this  is  a  diagnostic,  there  is  no  pass  or  fail  per  se.  The 
rater  should  be  honest  about  the  biases  and  skills  he/she  possesses 
and  should  emphasize  that  this  is  an  assessment  on  one  piece  of 
work. 

As  well,  the  rater  must  provide  the  interpreter,  based  on  the 
results  of  the  assessment,  with  a  suggested  plan  of  study.  This 
plan  should  include  a  brief  summation  of  the  strengths  and 
weaknesses  noted  in  the  interpretation  and  suggested  resources  and 
activities  the  interpreter  can  utilize  and  undertake  to  further 
develop  his/her  skills. 
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